Rational Agents and Environments

• Chapter 2 introduces intelligent agents and their environments.

Rational Agents

• Chapter 1 identified rational agents as central to AI.
• Introduces the concept of rationality, applicable to diverse agents and environments.
• Aims to develop design principles for building intelligent agents.
• Examines agents, environments, and their interaction.
• Rational agents behave optimally, considering environmental factors.
• Classifies environments to influence agent design.

Agents and Environments

• An agent perceives its environment through sensors and acts upon it via actuators.
• Human agents use organs for sensing and body parts for actions.
• Robotic agents use cameras/infrared sensors and motors.
• Software agents receive inputs like keystrokes/files and output displays/files.
• Agents can generally perceive their actions.
• A percept is the agent's input at any given moment.
• A percept sequence is the agent's complete history of perceptions.
• Agent's action choice depends on the entire percept sequence.
• Agent behavior is described by a function mapping percept sequences to actions.
• Agent function can be tabulated, but it's often large or infinite.
• Given an agent, its function can be constructed by observation.
• The agent function is an external view, while the agent program is its internal implementation.
• The vacuum-cleaner world example has two locations (A, B).
• Vacuum agent perceives location and dirt.
• Actions include moving left/right, sucking dirt, or doing nothing.
• One agent function: suck dirt if the current square is dirty, else move.

Good Behavior: The Concept of Rationality

• A rational agent does the right thing conceptually, filling the agent function table correctly.
• Performance measure evaluates agent success.
• Agents generate action sequences based on percepts, causing environment state changes.
• Desirable sequences indicate good performance.
• Performance measures are objective, imposed by the designer.
• In the vacuum-cleaner example, performance could be dirt cleaned in a shift.
• Rational agents maximize the set performance measure.
• A better performance measure rewards a clean floor.
• Design performance measures based on desired outcomes.
• "Clean floor" can be average cleanliness over time.
• Different agents can achieve the same average cleanliness differently.
• Choice of measure has philosophical implications.
• Rationality depends on the performance measure, prior knowledge, actions, and percept sequence.

Definition of a Rational Agent

• A rational agent selects actions that maximize expected performance, given evidence and knowledge.
• Depends on performance measure, environmental knowledge, sensors, and actuators.
• Claim: the agent described is rational; its expected performance is highest possible.
• Same agent can be irrational in different settings.
• Includes penalty for movement, cleaner only oscillates back and forth.
• Should also occasionally check and reclean if the squares can be dirty again.
• If geography is unknown, agent explores rather than sticking to specific squares.

Omniscience, Learning, and Autonomy

• Rationality differs from omniscience; rational agents maximize expected performance.
• Omniscience is impossible in reality and requires knowing the actual outcome of actions.
• Rationality maximizes expected performance, perfection maximizes actual performance.
• Rationality depends only on the percept sequence to date.
• Information gathering modifies future percepts.
• An agent gathers information through exploration.
• Agents should learn as much as possible from perceptions.
• Initial configuration has prior knowledge.
• Agents may not perceive or learn if environment is completely known a priori.
• Complete autonomy is seldom required initially.
• Acting randomly is required unless the designer gives assistance.
• Rational agent behavior becomes independent of knowledge after sufficient experience.
• Learning allows designing a single agent for various environments.

The Nature of Environments

• Task environments are the "problems" for rational agents to "solve".
• Specify the PEAS (Performance, Environment, Actuators, Sensors).
• Automated taxi driver is a complex example.
• Performance measures: correct destination, minimize cost, maximize safety/comfort/profits.
• Driving environment: roads, traffic, pedestrians, etc.
• Optional: operate in different locations, driving sides.
• Actuators: steering, accelerator, brake; output to passengers/vehicles.
• Sensors: cameras, speedometer, GPS, obstacle detectors; passenger input.
• Includes keyboard input/character output on screen.
• Key: Complexity of behavior -> percept sequence -> performance measure relationship.
• Some real environments are quite simple e.g. a robot inspecting partss on a conveyor belt.

Properties of Task Environments

• Task environments vary greatly in AI.
• Dimensions categorize task environments and determine suitable agent design.
• Environments can be fully or partially observable.
• Fully observable: agent's sensors access complete environment state.
• Effectively fully observable if relevant aspects are detectable.
• Useful because agents don't need to track the world.
• Partially observable: noisy, inaccurate sensors or missing data.
• Deterministic: the next state is fully determined by the current state and agent action.
• Stochastic: not deterministic.
• Agents in fully observable environments don't worry, no uncertainty.
• Partial observability often appears stochastic.
• Environments are deterministic from agent perspective even if truly stochastic.
• Strategic: the environment is deterministic, except for actions of other agents.
• Episodic: agent's experience is divided into atomic episodes, and the next episode does not depend on previous episodes or actions taken.
• Sequential: the current decision could affect all future decisions.
• Episodic tasks are simpler, because the agent does not need to "think ahead."
• Static: the environment does not change while the agent is deliberating.
• Dynamic: the environment can change during deliberation.
• Semidynamic: the environment itself does not change, but the agent's score does.
• Discrete: environment has a finite number of distinct states.
• Continuous: states, time, percepts, and actions take on a range of values.
• Single-agent: an agent solves a crossword puzzle by itself
• Multiagent: playing chess.
• Competitive environment: chess; players work against one another
• Cooperative environment: taxi driving; all agents avoid collisions

The Structure of Agents

• Agent = architecture + program.
• Environment classes and generators for testing agents.
• We need the AI agent program to implement the agent function that maps percepts to actions.
• Agent programs take the current percept as input and return an action.
• The action taken depends on the entire percept history.
• Table-driven agents use a lookup table of actions for percept sequences.
• The key challenge for Al is to produce rational behavior from small code sections

Simple Reflex Agents

• Actions are selected based on the current percept.
• Ignore the rest of the percept history.
• Based only on the current location and presence of dirt.
• The agent program is small compared to the lookup table.
• The number of possibilities are reduced from 4^t to 4 when ignoring the percept history.

Condition-Action Rule

• The program for a simple reflex agent is specific and inflexible.
• A general approach first builds a general-purpose interpreter.
• Interpreters process rules for a specific task environment.
• Simple reflex agents are simple but have limited intelligence.
• They work if the correct decision can be based on the current percept, only if the environment is fully observable.

Model-Based Reflex Agents

• The agent model is the most effective way to handle partial observability.
• Agents should keep track of the world they can't see now; maintain internal state.
• Internal states rely on the percept history.
• Updating the internal state requires two kinds of knowledge: how the world evolves and how agent actions affect the world.
• The model of the world can be implemented in simple circuits or scientific theories.
• An agent that uses such a model is a model-based agent.

Goal-Based Agents

• Knowing about the current state is not always enough to decide what to do.
• Also need the goal that is to be achieved.
• Agent must select actions to achieve the goal.
• Actions are based on current state descriptions and the goal information.

Utility-Based Agents

• Goals alone are not enough to generate high-quality behavior in environments.
• Goals have binary outcomes, but high-quality results need more comparison.
• A utility function helps measure agent performance.
• It also allows the agent to perform well when there are conflicting goals.
• The word "utility" refers to "the quality of being useful"

Learning Agents

• We want to give agents the ability to learn.
• This allows them to function in initially unknown environments.
• Agents can also surpass initial knowledge.
• The most important aspect of learning is that it will allow the agent to make improvements.
• Learning agents have a learning element, performance element, critic, and problem generator.
• The learning element improves, and the performance element selects actions.
• Feedback from the critic is on how the agent is doing; modifies it to do better.
• The problem generator helps get new, informative experiences.
• The performance standards are external, and should not be modified.