In the previous week’s post, we discussed the nature of AI agents and what constitutes rationality in them. However we did not dive into the nature of the environment these agents function in. This environment plays a crucial role in defining the goals and objectives for the agents and thus what they learn.

The environment can be simply defined as the world the agent functions in; for humans it is earth, for chess pieces it is the chessboard and so on. The agent has to play by the rules of the environment to achieve its goal. The rules of the chessboard govern the movement of the pieces, a bishop has to go diagonally while the king will always move only one step (except castling, I know).

Parts of the Agent

We already know what an Agent is, it is the system which acts on the environment. A real world agent therefore will need to understand or visualize the environment and perform actions according to it. Sensors play the role of visualizing or ‘seeing’ the environment, it reads the necessary data from the environment and provides it to the agent. On the other hand, we have Actuators which perform actions on the environment, thus changing the state of it. In a real world system, we often have motors etc. which act as the actuators.

Finally we have the environment. Environment defines the ‘terrain’ in which the agent is expected to function in. In case of a self-driving car, the environment includes urban and rural roads, traffic signals, pedestrians etc. The rules of the environment are the general traffic rules that the driver is expected to follow. Any decision made by the agent outside of these rules is considered illegal.

Nature of Environment

Not all environments are the same. In case of a chessboard, a player can visualize the whole board, giving it an omniscient view. Elsewhere in case of maze, we only know the paths available in front of us and not all of them (which would make the maze pointless). This differentiation classifies the environment based on Fully Observable vs Partially Observable, something we got into in the last post.

We can also classify environment based on the number of agents acting on it, Single Agent vs Multi Agent. Maze is an example of single agent system, where one entity is trying to get out of the maze, while chess is multiagent, where two players are competing against each other to win the game.

Another classification is whether the environment is Deterministic vs Non-Deterministic. If the changes in the environment is based purely on the current state of the environment and the action of the agent, then it is termed as Deterministic. On the other hand, if the changes in the environment can be made by an external force, it is termed as Non-Deterministic or Stochastic.

Table 1. Classification of various environments from Russel et al.

There are many such ways to classify an environment, discussed well in the Russel and Norvig’s book “Artificial Intelligence: A Modern Approach”.

Conclusion

AI environment plays a crucial in determining the action space of an Agent and the algorithm used to train it. We must be aware of the rules and the boundaries of the environment while designing the agent. Now that we are aware of the functionalities of an Agent and the nature of Environment, we can focus on another aspect that matters, namely Performance.