Intelligent Agents: Wooldridge Ch. 2, Russell Ch. 2

• Lecture topic: Intelligent Agents, covering basic concepts and architectures
• Reading material includes: Wooldridge, Chapter 2, and Russell & Norvig, Chapter 2

Agents and Environments

• An agent perceives its environment through sensors and acts upon it via actuators
• A percept is an agent's perceptual input at a given instant
• A percept sequence comprises the complete history of what the agent has perceived
• The agent function maps a percept sequence to an action, described abstractly
• The agent program runs on physical architecture to produce the function

Rationality

• An ideal rational agent should take actions that maximize its performance measure, based on the percept sequence to date
• Performance measure is an objective criterion for the success of an agent's behavior
• It evaluates the agent's performance across previous environment states
• Rationality is about expected success, considering the agent's perceptions

Key Aspects of Rationality

• The performance measure defines success
• An agent's prior knowledge of the environment is relevant
• The actions that the agent can perform are important
• The agent's percept sequence to date matters

Example of Rationality in Chess

• Performance measure: winning the game
• Knowledge: rules of chess
• Actions: legal moves
• Percepts: board state

Autonomy

• Rational agents should be autonomous
• An autonomous agent's behavior should be determined by its own experience
• Agents can gain autonomy through learning
• Initially, agents need built-in knowledge and the capacity to learn

Task Environment

• Task Environment includes all relevant external factors and conditions that impact an agent's behavior and ability to perform

Specifying The Task Environment with PEAS

• Performance, Environment, Actuators, Sensors include relevant external factors impacting behavior
• Performance (Goal Context): Considers surroundings relevant to the agent's task
• Guides the agent to pursue its objectives

Environment

• The external world the agent interacts with, which can be physical or virtual

Actuators(Interaction)

• The agent interacts with the task environment through actions

Sensors(Agent's Perspective)

• Includes everything the agent can sense and potentially act upon

Agent Architectures

• An agent's architecture combines a program with a physical device, and is increasing in complexity
• reactive, deliberative, hybrid, and learning architectures

Reactive Architectures (Behavior-Based Architectures)

• Core principle is direct mapping from perceptions to actions, focusing on immediate responses

Characteristics of Reactive Architectures

• Simplicity: Relatively easy to design and implement
• Responsiveness: Fast reaction times, suited for dynamic environments
• Limited Reasoning: Lacks internal world models, planning, or complex reasoning
• Emergent Behavior: Complex behavior emerges from simple reactive behaviors

Subsumption Architecture

• Organizes behaviors in layers, where higher layers override or "subsume" lower layers
• Each layer is a simple perception-action rule, prioritizing basic survival behaviors first

Behavior-Based Robotics

• Uses collections of simple behaviors
• Often inspired by biological systems

Deliberative Architectures (Symbolic AI Architectures)

• Involves thoughtful and planned action, where agents carefully consider and plan before acting

Reasoning and Planning

• Deliberative agents use reasoning and planning to make decisions by:
  • Analyzing the current situation
  • Predicting the consequences of different actions
  • Evaluating options based on goals and knowledge

Key Aspects of Deliberative Agents

• Internal Models: Rely on internal models of the world to represent their environment
• Knowledge-Based: Utilize knowledge representation and logical inference for decision-making

Model-Based Agents

• Has an internal representation of the world, often called a "model"
• Reason about their environment by keeping track of the world
• Predict future states by evaluating how the word works
• Makes more informed decisions

Logic-Based Agents

• It implements simple logic-based systems
• It implemented Probabilistic systems, e.g., Bayesian networks, hidden Markov models
• It includes state transition diagrams, data structures, and any other form of representation that captures the essential aspects of the environment

Goal-Based Agents

• Have explicit goals they are trying to achieve
• Need to know the current state and the goal state to make decisions
• The goals can be simple or complex

Advantages of Goal-Based Agents

• They are designed to achieve specific objectives
• It can solve complex problems that require planning and reasoning
• It can adapt to changing environments by replanning if necessary

Disadvantages of Goal-Based Agents

• Planning and search can be computationally expensive
• Definig goals in a way that is understandable and usable by an agent can be challenging
• They struggle in environments where there have incomplete or inaccurate information

Utility-Based Agents

• Use a utility function to measure the happiness of a state
• They try to maximize their expected utility, making more rational decisions than goal-based agents
• Suited when a situation has multiple and conflicting goals

Discounting

• Maximize the sum of rewards
• Prefer rewards now to rewards later
• Has a policy that represents the choice of action for each state
• Utility: sum of discounted rewards

Advantages of Utility-Based Agents

• Make decisions that maximize their expected utility
• Can handle uncertainty by considering probabilities and expected values
• Handles complex preferences

Disadvantages of Utility-Based Agents

• Defining a utility function that accurately reflects the agent's preferences can be challenging
• Calculating expected utility can be computationally expensive
• Utility functions can be subjective

Belief-Desire-Intention (BDI) Architecture

• It is a popular architecture of rational agents
• Based on the agent has about the world
• Goals and objectives the agent wants to accieve
• Desires the agents has committed to achieving

BDI Cycle

• Observation: Agent perceives its environment
• Belief update: Agents update beliefs based on observations
• Deliberation: Agent selects desires it has to execute
• Means-End reasoning: Selects plans to achieve intentions
• Intention Update: updates intentions
• Action Execution: Executes based on current intentions

BDI Architecture Advantages

• Has a way to represent rational agents
• Can handle complex goals and situations
• Suited for multi-agent systems
• It can be computationally expensive and requires careful design and implementation

Deliberative Agents

• Reason about the world, plan an action, and use a model of the world
• They can be goal or utility-based
• Are slow but can make complex decisions

Reactive Agents

• React to the current input
• Fast but has limited capability

Hybrid Architectures

• Combine the advantages of both deliberative and reactive architectures
• Often use a layered approach
• Involves a reactive layer that handles immediate responses, and a deliberative layer for planning and reasoning

Hierarchical Architectures

• Have multiple layers, with greater responsibility for more abstract reasoning and planning
• Communication flows in both directions, higher and lower, and vise versa

Subsumption architectures

• Uses a bottom-up approach that start with simple reaction and gradually add complex layers
• Higher layers subsume and override the behavior of lower layers