Lecture 2 questions

Intelligent Agents: Basic Concepts and Architectures

• A lecture given by Dr. Mustafa Shiple (NTI)

Lecture Outline

• Rationality and Rational Agents
• Agent Architectures
• Reactive Architectures
• Deliberative Architectures
• Hybrid Architectures
• Learning Architectures via Reinforcement and Deep Learning

Agents and Environments

• An agent perceives its environment through sensors, acting upon it through actuators.
• Percept: An agent's perceptual inputs at a specific moment.
• Percept Sequence: A complete historical record of everything an agent has ever perceived.
• Agent Function: Maps percept sequences to actions, described abstractly.
• Agent Program: Executes on physical architecture to implement the agent function.

Rationality

• An ideal rational agent chooses actions maximizing performance, based on percept sequence history.
• Performance Measure: Objective criterion defining successful agent behavior.
• Rationality: Achieving anticipated success based on acquired perceptions.

Key Aspects of Rationality

• The performance measure dictates success.
• The agent's existing knowledge impacts success.
• The actions that the agent can perform impacts success.
• Success depends on the agent's accumulated percept sequence.
• In a chess-playing example:
  • Winning the game constitutes the performance measure.
  • Knowledge comprises the rules of chess.
  • Actions are legal moves.
  • The percept is the board state.

Autonomy

• Rational agents should be autonomous.
• Autonomous behavior arises from an agent's own experiences.
• Agents gain autonomy through learning.
• Initially, agents need some innate knowledge plus the ability to learn.

Task Environment

• Encompasses external factors and conditions affecting behavior and performance.

Specifying the Task Environment Using PEAS

• PEAS defines Performance, Environment, Actuators, and Sensors
• PEAS encompasses factors affecting behavior and performance. Includes:
  • Performance (Goal Context): Focuses on surroundings relevant to an agent's tasks, structuring the environment.
  • Environment (Surroundings/World): External context the agent interacts with, physically or virtually.
  • Actuators (Interaction): Mechanisms for interaction with the environment.
  • Sensors (Agent's Perspective): What the agent perceives and acts upon.

Agent Architectures

• An agent's architecture merges a program with a physical device.
• Architectures vary in complexity and capability.

Agent Architectures: Reactive

• Reactive Architecture (Behavior-Based Architecture)
• Simple Reflex Agents:
  • Core principle: Direct mapping from perceptions to actions, focusing on immediate environmental responses.
  • Simplicity: Easy to design and implement.
  • Responsiveness: Quick reactions, suitable for dynamic environments.
  • Limited Reasoning: Lacks complex reasoning or planning.
  • Emergent Behavior: Simple reactive behaviors interact into complex behavior.

Subsumption Architecture

• An example of Reactive Architecture
• Behaviors are organized in layers, where higher layers override lower layers.
• Layers execute simple action rules.
• Prioritizes basic survival above all else.

Behavior-Based Robotics

• Employ collections of simple, biologically-inspired behaviors.

Potential Fields

• Agents navigate by reacting to attractive/repulsive "forces" from goals/obstacles.

Simple Reflex Agents: Pros and Cons

• Pros:
  • Simplest agent type.
  • Chooses actions based on current perception alone, ignoring history.
  • Its function relies on condition-action rules; if a condition exists, then take an action.
• Cons:
  • Possesses limited intelligence.
  • Reacts only to the present.
  • Struggles in partially observable environments; cannot learn or adapt to new circumstances.
  • Interactions amongst behaviors quickly become too difficult to manage.
  • Is a short-term reactive system in a local setting.

Deliberative Architectures

• (Symbolic AI Architectures): Involves reasoned decision making.
  • A. Model-Based Agents,
  • I. Logic-Based agents.
  • B. Goal-Based Agents.
  • C. Utility-Based Agents.
  • D. Belief-Desire-Intention

Considerations for Deliberative Architectures

• Thoughtful and Planned Action through careful consideration and planning.
• Reasoning and Planning: Using reasoning and planning to make decisions.
  • Analyzing current situation.
  • Predicting the consequences of actions.
  • Evaluating actions based on goals and knowledge.
• Internal Models: Internal world models to represent their environment.
• Knowledge-Based: Employs knowledge representation and logical inference to make decisions.

Model-Based Reflex Agents

• Employs an internal "model" of the world that allows:
  • "Reasoning About its Environment" (Keep Track of the World).
  • Predictive Future states.
  • Informed decisions.
• Implemented in:
  • Simple logic-based system (cannot handle Uncertainty)
  • Probabilistic (Bayesian networks, hidden Markov models) (to handle Uncertainty)
  • State Transition Diagrams
  • Data Structures
  • Any other form of representation that captures the essential aspects of the environment

Logic-Based Agents

• Implemented in first-order logic equations.

Goal-Based Agents

• Possess explicit goals.
• Need current and goal states to make decisions.
• Goals can be simple/complex, represented in logical formulas, state descriptions, or utility functions.

Goal-Based Agents: Pros and Cons

• Pros:
  • Goal-Oriented Behavior: Designed to achieve aims.
  • Planning and Problem-Solving: Capable of handling intricate challenges via planning/reasoning.
  • Flexibility: Ability to adapt by replanning.
• Cons:
  • Computational Cost: Expensive planning and searching.
  • Goal Representation: Challenges in forming useful goal definitions.
  • Incomplete Information: Struggles with inadequate or inaccurate data.

Utility-Based Agents

• Measures the ''happiness'' of a state.
• Maximizes expected utility for rational decisions, especially with several conflicting goals.
• Agent function is based on percept internal state goal and utility function.

Example of Utility-Based Agents

• In a self-driving car setting:
  • Arrival time (earlier is better).
  • Safety (fewer accidents are better).
  • Fuel efficiency (less fuel consumption is better).
  • Passenger comfort (smoother ride is better).

Discounting

• Maximizes the sum of rewards
• Prefers rewards now to rewards later
• Value of rewards decay exponentially

Advantages of Utility-Based Agents

• Advantages:
  • Rational Decision-Making: Makes ideal, utility maximizing decisions.
  • Handling Uncertainty: Considers probabilities and predicted values.
  • Complex Preferences: Balances conflicting goals.
• Disadvantages:
  • Defining the Utility Function: Struggle to develop representative functions.
  • Computational Cost: Costly utility calculations, especially in complex settings.
  • Subjectivity: Tendency of utility functions towards subjectivity agents.

Belief-Desire-Intention (BDI) Architecture

• A rational agent architecture based on:
  • Beliefs: Agent information concerning the world.
  • Desires: Preferred agent objectives.
  • Intentions: Goals committed to achieving.
• BDI Cycle
  • Observation: Agent perceives the environment.
  • Belief Update: Agent updates beliefs based on observations.
  • Deliberation: Agent selects desires to become intentions.
  • Means-Ends Reasoning: Agent selects plans to achieve intentions.
  • Intention Update: Agent updates intentions.
  • Action Execution: Agent executes actions according to intentions.

Components of BDI Architecture

• Belief Revision: Determines a new set of beliefs with perceptual input and the agent's current beliefs.
• Belief DB: A set of current beliefs, representing information the agent has about the agent's current environment.
• Generate Options: Determines the options available to the agent (its desires), based on its current beliefs about its environment and its current intentions.
• Desires: Representing possible courses of actions available to the agent.
• Filter: Represents the agent's deliberation process, and determines the agent's intentions based on the agent's current beliefs, desires, and intentions:
  • Drop Intentions that are no longer achievable if the expected cost exceeds the expected gain.
  • Retain any not achieved and has positive overall benefit.
  • Retain intentiions that exploit new opportunities.
• Intentions: Represents the agent's current focus that is selected from goals set (intentions) which are committed to be achieved.

Advantages and Disadvantages of BDI Architecture

• Advantages
  • Intuitive and natural way to represent rational agents.
  • Can handle complex goals and situations.
  • Well-suited for multi-agent systems.
• Advantages
  • Can be computationally expensive.
  • Requires careful design and implementation

Deliberative vs. Reactive Architectures

• Deliberative Agents:
  • Reason about the world, plan actions, and use a complex world model.
  • Goal-based or utility-based.
  • Slower, but makes complex decisions.
• Reactive Agents:
  • React to the current percept.
  • Do not use a model of the world.
  • Simple reflex agents execute actions fast, but limited in capability.

Hybrid Architectures

• Combine the advantages of both deliberative and reactive architectures.

Hierarchical Architectures

• An example of Hybrid Architecture
• Multiple layers, with higher layers responsible for more abstract reasoning/planning tasks.
• Lower layers handle reactive behaviors.
• Two-way communication channels.
• High-Level (BDI Layer)
  • Manages beliefs about the office layout, package locations, and delivery schedules.
  • Uses of desires (e.g. "Deliver package to john") and intentions (e.g., "Go to john's office now") to plan and make decisions. Uses a path-planning algorithm to determine the best route.
  • Mid-Level (Navigation Layer): Receives navigation commands from the high-level layer (e.g., "go to hallway 3").
  • Handles local navigation, obstacle avoidance, and adjusting the robot's path based on sensor data.
  • Low-Level (Motor Control Layer): Receives motor commands from the navigation layer and controls the robot's motors to move it forward, turn, and stop.

Subsumption Architecture

• Is a type of layered architecture
• Uses a bottom-up approach, starting with simple reactive behaviors and gradually adding more complex layers.
• Avoids obstacles by detecting objects and moves the robot to avoid them.
• Keeps the robot moving down hallways, following the walls.
• Directs the robot towards the delivery destination/package pickup location
• Higher layers can "subsume" or override the behavior of lower layers.

Hierarchical vs. Subsumption Architectures

• Control Flow:
  • Hierarchical is top-down.
  • Subsumption is bottom-up.
• Centralization:
  • Hierarchical is centralized.
  • Subsumption is decentralized.
• Planning vs. Reactivity:
  • Hierarchical balances planning with reactive behavior.
  • Subsumption prioritizes reactivity.
• BDl Suitability:
  • Hierarchical is better suited for BDI integration.
  • Subsumption requires a hybrid approach.