AI: Agents and Environments

Lecture 2: Agents and Environments

• Study notes for the lecture on Agents and Environments is based on Chapter 2 of "Artificial Intelligence: A Modern Approach" by Stuart Russell and Peter Norvig
• The topics cover Agents and Environments, Rational Agents, PEAS (Performance measure, Environment, Actuators, Sensors), Environment types and Agent types

Agents

• An agent in AI is any entity that perceives its environment through sensors and acts upon it through actuators (or effectors).
• Agents can be software-based (e.g., chatbots) or physical (e.g., robots).
• Agents perceive their environment through sensors and affect their environment through actuators.

Agents and environments

• Perception often refers to the ability of machines to interpret and understand data from the environment.
• This can include tasks such as computer vision, speech recognition, and sensor data processing, like in self-driving cars.
• Feedback from the environment impacts the agent's future actions.

Agent Function

• The agent function is a mathematical mapping from percept histories (all past and present perceptions) to actions.
• It defines how an agent should behave in every possible situation.
• It is an abstract concept that specifies only what the agent should do, not how it is implemented.
• Represented as f : P* → A, where P* is all possible percept histories and A is the set of possible actions.

Agent Program

• The agent program is the concrete implementation of the agent function
• The program is a piece of software that runs on a physical or virtual agent, determining its actual behavior
• It takes percepts as input, processes them, and outputs an action.
• Agent program l runs on a machine M to implement f, so f = Agent(l,M)
• Real machines have limited speed and memory, so the agent function f depends on M as well as l

Agent Function and Program Example

• A vacuum-cleaner agent can decide whether to move or clean based on sensor input
• The agent function could advise to clean if the location is dirty, and move to a new location if the location is clean
• The agent program would be the actual code implementing this logic in Python, Java, etc.
• In a vacuum world, percepts could include the location and status e.g. [A,Dirty].
• Possible actions are Left, Right, Suck and NoOp.
• Agent Function example is: if [A, Clean] then Right, if [A, Dirty] then Suck, if [B, Clean] then Left, if [B, Dirty] then Suck etc
• Agent Program example is:
  • function Reflex-Vacuum-Agent([location,status]) returns an action, if status = Dirty then return Suck, else if location = A then return Right, else if location = B then return Left

Rational Agent

• Rationality is the state of being reasonable and having good judgment.
• A rational agent is one that does the right thing, always taking the best possible action to maximize its performance based on available information.
• It acts in a way that is expected to achieve the best outcome, or when uncertainty is involved, the best expected outcome.
• Performance measure is usually chosen by the Agent designer
• A rational agent chooses an action that maximizes its expected performance, given the percept sequence, knowledge it has about the environment, possible actions it can take, and the performance measure that evaluates success.
• A self-driving car is an example of a rational agent that would take actions that minimize travel time while ensuring safety and obeying traffic laws
• A chess-playing AI is a rational agent that selects moves to maximize the probability of winning based on the current board state
• Rational agents must make the best decisions based on available information and learn over time
• They can make decisions based on their current knowledge and beliefs, but they cannot predict the future, needing to deal with uncertainty
• It is essential for rational agents to explore and learn in unknown environments
• Real rational agents may make mistakes, so a goal of rationality is to minimize the mistake, given constraints
• To make effective decisions, a rational agent perceives its environment and processes information for autonomous action

Task Environment

• The task environment of an AI agent is the external world in which the agent operates and interacts
• It defines everything that affects the agent's decision-making and performance, and it's crucial for designing intelligent agents

PEAS Framework

• A task environment is often described using the PEAS framework:
  • Performance Measure: Criteria for evaluating the agent's success.
  • Environment: The external world in which the agent operates.
  • Actuators: The means by which the agent takes actions.
  • Sensors: The means by which the agent perceives the environment.
• Automated taxi system example
  • Performance measure: Safety, destination, profits, legality, comfort, etc
  • Environment: City streets, freeways; traffic, pedestrians, weather, etc
  • Actuators: Steering, brakes, accelerator, horn
  • Sensors: Camera, sonar, radar, GPS, engine sensors, microphone
• Medical diagnosis system example
  • Performance measure: patient health, cost, reputation
  • Environment: Patients, medical staff, insurers
  • Actuators: Screen display, email (questions, tests, diagnoses, treatments, referrals)
  • Sensors: keyboard/mouse(entry of symptoms, findings, patient's answers)
• Agent is a Part-picking robot
  • Performance measure: Percentage of parts in correct bins
  • Environment: Conveyor belt with parts, bins
  • Actuators: Jointed arm and hand
  • Sensors: Camera, joint angle sensors

Sample Task Environments

• Self-Driving Car
  • Performance Measure: Safety, speed, traffic rules, passenger comfort
  • Environment: Roads, traffic, pedestrians, weather
  • Actuators: Steering, accelerator, brakes
  • Sensors: Cameras, LiDAR, GPS, speed sensors
• Chess AI
  • Performance Measure: Winning the game
  • Environment: Chessboard
  • Actuators: Moving pieces
  • Sensors: Board state, opponent moves
• Vacuum Cleaner
  • Performance Measure: Cleanliness of the floor, battery usage
  • Environment: Room with dirt
  • Actuators: Wheels, suction motor
  • Sensors: Dirt sensor, position sensor

Environment Types

• Deterministic vs. Stochastic
  • Deterministic: Next state is entirely determined by the current state and action, such as Chess
  • Stochastic: Outcomes involve randomness, like Poker
• Fully Observable vs. Partially Observable
  • Fully Observable: Agent has complete knowledge of the environment, like Chess
  • Partially Observable: Some information is hidden or uncertain, like self-driving cars with limited visibility due to fog
• Static vs. Dynamic
  • Static: Environment does not change while the agent is deciding, such as Sudoku puzzles
  • Dynamic: Environment evolves over time, such as Stock market predictions.
• Discrete vs. Continuous
  • Discrete: Finite number of possible states/actions, such as Tic-Tac-Toe
  • Continuous: Infinite range of states/actions, such as Robot arm movement
• Single-Agent vs. Multi-Agent
  • Single-Agent: Only one intelligent entity acts, such as Pathfinding in a maze.
  • Multi-Agent: Multiple agents interact and compete or cooperate, such as Multiplayer online games
• Episodic vs. Sequential: An agent's action is divided into atomic episodes, and decisions do not depend on previous decisions/actions.
• Known vs. Unknown: An environment is "known" if the agent understands the laws that govern the environment's behaviour

Summary of Environment Types

• Fully Observable: Agent has complete knowledge of the environment at all times e.g. Chess, Sudoku Solver
• Partially Observable: Agent has limited perception and must infer missing information e.g. Self-Driving Car, Poker
• Deterministic: The next state is completely predictable e.g. Chess, Arithmetic Calculator
• Stochastic: The next state has randomness and uncertainty e.g. Stock Market Prediction, Poker
• Episodic: Each action is independent of previous actions e.g. Image Classification, Spam Detection
• Sequential: Each action affects future actions and outcomes e.g. Chess, Self-Driving Car
• Static: The environment does not change while the agent is deciding Turn-based Board Games, Crossword Puzzles
• Dynamic: The environment can change in real time even if the agent does nothing e.g. Real-time Video Games, Self-Driving Cars
• Discrete: The number of possible states and actions is finite e.g. Tic-Tac-Toe, Chess
• Continuous: The number of possible states and actions is infinite e.g. Autonomous Drones, Robot Arm Control
• Single-Agent: The agent operates alone without competing or cooperating entities e.g. Maze-Solving Robot, Weather Prediction
• Multi-Agent: Multiple agents interact competing or cooperating e.g. Soccer-playing Robots, Online Auctions

Agent Types

• Table driven Agent
• Simple reflex agents
• Model-based reflex agents
• Goal-based agents
• Utility-based agents
• Learning Agent

Table-Driven Agent

• It is one of the simplest types of agents in AI, storing predefined responses for every possible percept sequence
• It operates by using a lookup table that maps percept histories to actions.
• Relatively simple and easy to implement for problems with a manageable number of states and actions
• Exponential Growth: The table size increases exponentially with the number of percepts
• Lack of Adaptability: Cannot handle unseen situations or learn from experience.
• Inefficient Memory Usage: Storing all possible percept sequences requires significant memory

Simple Reflex Agents

• These agents operate based on a simple "if-then" rule format, taking actions based on the current percept or input
• It does not consider past states or future consequences
• An internal model of the environment is maintained to handle partial observability in self-driving cars using stored maps to navigate with temporary obstructions.
• Limitations include No Memory, No Long-Term Planning, Inefficiency

Model-Based Reflex Agents

• These agents maintain an internal model or representation of the world
• It makes decisions by considering past states, current percepts, and anticipated future states
• Advantages are: agent does not repeat actions unnecessarily, handles Partial Observability, avoids redundant movement
• examples include Smart Home Cleaning Robots and AI in Video Games
• Disadvantages are that this method can be computationally expensive, Models may not capture the real-world, Models cannot anticipate all potential situations, Need for frequent updates, pose interpretation

Goal-Based Agents

• Goal-based agents have predefined goals or objectives that guide their decision-making process
• They take actions that are expected to move them closer to achieving their goals.
• Goals can be more complex in that efforts have to be made to find a way to achieve goals such as getting to a hospital
• Search and planning happen when you find the path to the goal state
• The agent is limited to specific and unadaptable, and the agent is ineffective for complex tasks that have too many variables
• Significant domain knowledge is required to define goals

Utility-Based Agents

• Utility-based agents evaluate the utility or desirability of different actions and choose actions that maximize their expected utility or reward
• This helps them deal with complex and uncertain situations adaptively.
• They often used in applications where they have to compare and select among multiple options
• They should map their state to a real value ("am I happy").
• They can trade off immediate gains vs future, risk vs reward. Some solutions could be better than others which is given thanks to a utility function

Learning Agents

• Learning agents can adapt and improve their behavior over time through learning mechanisms.
• They acquire knowledge and skills from experience, feedback, and training data
• The agent would improves over time by monitoring its performance and suggesting better modeling and setting new rules.
• The learing agent has four stages -Learning Element to make improvements -Critque that provides learning element with its feedbacks -Performance Element selects external actions -Problem Generator that suggest to create better, more informative elements

Summary

• An agent interacts with an environment through sensors and actuators.
• A task is defined by PEAS descriptions/specifications
• The more difficult the environment, the more complex agent designs and representations required
• Rational agents choose actions to maximize their utility
• The agent function, implemented by an agent program, runs on a machine, and the function describes what the agent does in all circumstances

Agent Types Summary

• Simple Reflex Agents react without memory
• Model-Based Reflex Agents remember the past but don't think ahead
• Goal-Based Agents have a target but efficiency isn't measured
• Utility-Based Agents choose via a scoring system
• Learning-Based Agents improving via past experience