Intelligent Agents PDF

Intelligent Agents  Outline  Agents and environments  Rationality  PEAS (Performance measure, Environment, Actuators, Sensors)  Environment types  Agent types  Agents  An agent is anything that can be viewed as perceiving its environment throu...

Intelligent Agents  Outline  Agents and environments  Rationality  PEAS (Performance measure, Environment, Actuators, Sensors)  Environment types  Agent types  Agents  An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators  Human agent: eyes, ears, and other organs for sensors; hands, legs, mouth, and other body parts for actuators  Robotic agent: cameras and infrared range finders for sensors; various motors for actuators  Agents and environments  The agent function maps from percept histories to actions:  [f: P*  A]  The agent program runs on the physical architecture to produce f  agent = architecture + program  Vacuum-cleaner world  Percepts: location and contents, e.g., [A,Dirty]  Actions: Left, Right, Suck, No Op A vacuum-cleaner agent  Rational agents  An agent should strive to "do the right thing", based on what it can perceive and the actions it can perform. The right action is the one that will cause the agent to be most successful  Performance measure: An objective criterion for success of an agent's behavior  E.g., performance measure of a vacuum-cleaner agent could be amount of dirt cleaned up, amount of time taken, amount of electricity consumed, amount of noise generated, etc.   What is rational at any given time depends on four things:  The performance measure that defines the criterion of success.  The agent’s prior knowledge of the environment.  The actions that the agent can perform.  The agent’s percept sequence to date  Rational agents  Rational Agent: For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has.   Rational agents  Rationality is distinct from omniscience (all- knowing with infinite knowledge)  Agents can perform actions in order to modify future percepts so as to obtain useful information (information gathering, exploration)  An agent is autonomous if its behavior is determined by its own experience (with ability to learn and adapt)  PEAS  PEAS: Performance measure, Environment, Actuators, Sensors  Must first specify the setting for intelligent agent design  Consider, e.g., the task of designing an automated taxi driver:  Performance measure  Environment  Actuators  Sensors  PEAS  Must first specify the setting for intelligent agent design  Consider, e.g., the task of designing an automated taxi driver:  Performance measure: Safe, fast, legal, comfortable trip, maximize profits  Environment: Roads, other traffic, pedestrians, customers  Actuators: Steering wheel, accelerator, brake, signal, horn  Sensors: Cameras, sonar, speedometer, GPS, odometer, engine sensors, keyboard  PEAS  Agent: Medical diagnosis system  Performance measure: Healthy patient, minimize costs, lawsuits  Environment: Patient, hospital, staff  Actuators: Screen display (questions, tests, diagnoses, treatments, referrals)  Sensors: Keyboard (entry of symptoms, findings, patient's answers)  PEAS  Agent: Part-picking robot  Performance measure: Percentage of parts in correct bins  Environment: Conveyor belt with parts, bins  Actuators: Jointed arm and hand  Sensors: Camera, joint angle sensors  PEAS  Agent: Interactive English tutor  Performance measure: Maximize student's score on test  Environment: Set of students  Actuators: Screen display (exercises, suggestions, corrections)  Sensors: Keyboard  Environment types  Fully observable (vs. partially observable): An agent's sensors give it access to the complete state of the environment at each point in time.  Fully observable environments are convenient because the agent need not maintain any internal state to keep track of the world.  An environment might be partially observable because of noisy and inaccurate sensors or because parts of the state are simply missing from the sensor data—for example, a vacuum agent with only a local dirt sensor cannot tell whether there is dirt in other squares, and an automated taxi cannot see what other drivers are thinking.  If the agent has no sensors at all then the environment is unobservable.  Single agent vs. multiagent:  The distinction between single-agent and multiagent environments may seem simple enough. For example, an agent solving a crossword puzzle by itself is clearly in a single-agent environment, whereas an agent playing chess is in a two agent environment.  For example, in chess, the opponent entity B is trying to maximize its performance measure, which, by the rules of chess, minimizes agent A’s performance measure. Thus, chess is a competitive multiagent environment.  In the taxi-driving environment, on the other hand, avoiding collisions maximizes the performance measure of all agents, so it is a partially cooperative multiagent environment.  The agent-design problems in multiagent environments are often quite different from those in single-agent environments;  for example, communication often emerges as a rational behavior in multiagent environments; in some competitive environments, randomized behavior is rational because it avoids the pitfalls of predictability.  Deterministic vs. stochastic  If the next state of the environment is completely determined by the current state and the action executed by the agent, then we say the environment is deterministic; otherwise, it is stochastic.  In principle, an agent need not worry about uncertainty in a fully observable, deterministic environment.  If the environment is partially observable, however, then it could appear to be stochastic.  Most real situations are so complex that it is impossible to keep track of all the unobserved aspects; for practical purposes, they must be treated as stochastic.  Eg: Taxi driving  Episodic vs Sequential Environment  Episodic (vs. sequential): The agent's experience is divided into atomic "episodes" (each episode consists of the agent perceiving and then performing a single action), and the choice of action in each episode depends only on the episode itself.  In sequential environments, on the other hand, the current decision could affect all future decisions. Chess and taxi driving are sequential: in both cases, short-term actions can have long-term consequences.  Episodic environments are much simpler than sequential environments because the agent does not need to think ahead.  Static vs. dynamic  If the environment can change while an agent is deliberating, then we say the environment is dynamic for that agent; otherwise, it is static.  Static environments are easy to deal with because the agent need not keep looking at the world while it is deciding on an action, nor need it worry about the passage of time.  If the environment itself does not change with the passage of time but the agent’s performance score does, then we say the environment is semidynamic.  Examples  Taxi driving is clearly dynamic:  the other cars and the taxi itself keep moving while the driving algorithm dithers about what to do next.  Chess, when played with a clock, is semidynamic.  Crossword puzzles are static  Discrete vs. continuous  The discrete/continuous distinction applies to the state of the environment, to the way time is handled, and to the percepts and actions of the agent.  For example, the chess environment has a finite number of distinct states (excluding the clock).  Chess also has a discrete set of percepts and actions.  Taxi driving is a continuous-state and continuous- time problem.  Known vs. Unknown  Strictly speaking, this distinction refers not to the environment itself but to the agent’s (or designer’s) state of knowledge about the “laws of physics” of the environment.  In a known environment, the outcomes (or outcome probabilities if the environment is stochastic) for all actions are given. Obviously, if the environment is unknown, the agent will have to learn how it works in order to make good decisions.  Environment types Chess with Chess without Taxi driving a clock a clock Fully observable Yes Yes No Deterministic Strategic Strategic No Episodic No No No Static Semi Yes No Discrete Yes Yes No Single agent No No No  The environment type largely determines the agent design  The real world is (of course) partially observable, stochastic, sequential, dynamic, continuous, multi-agent. Task Environments and their characteristics  Agent functions and programs  Job of AI is to design an agent program that implements the agent function - the mapping from percepts to actions.  We assume this program will run on some sort of computing device with physical sensors and actuators—we call this the architecture.  agent = architecture + program.  One agent function (or a small equivalence class) is rational  Aim: find a way to implement the rational agent function concisely. Table Driven Agent  Table-driven agent  Drawbacks:  Must construct a table that contains the appropriate action for every possible percept sequence  Huge table  Take a long time to build the table  No autonomy  Even with learning, need a long time to learn the table entries Agent program for a vacuum-cleaner agent  Agent types  Four basic types in order of increasing generality:   Simple reflex agents  Model-based reflex agents  Goal-based agents  Utility-based agents Simple reflex agents  Simple Reflux agent  The simplest kind of agent is the simple reflex agent. These agents select actions on the basis of the current percept, ignoring the rest of the percept history.  Example: The vacuum agent  “The car in front is braking.” Then, this triggers some established connection in the agent program to the action “initiate braking.”  a connection as a condition–action rule,  if car-in-front-is-braking then initiate-braking. Simple reflex agents  The INTERPRET-INPUT function generates an abstracted description of the current state from the percept,  and the RULE-MATCH function returns the first rule in the set of rules that matches the given state description.  will work only if the correct decision can be made on the basis of only the current percept—that is, only if the environment is fully observable. Model-based reflex agents  Model-based reflex agents  The most effective way to handle partial observability is for the agent to keep track of the part of the world it can’t see now.  That is, the agent should maintain some sort of internal state that depends on the percept history and thereby reflects at least some of the unobserved aspects of the current state.  Model-based reflex agents  Updating this internal state information as time goes by requires two kinds of knowledge to be encoded in the agent program.  First, we need some information about how the world evolves independently of the agent  For example, that an overtaking car generally will be closer behind than it was a moment ago.  Second, we need some information about how the agent’s own actions affect the world  For example, that when the agent turns the steering wheel clockwise, the car turns to the right.  This knowledge about “how the world works”— whether implemented in simple Boolean circuits or in complete scientific theories—is called a model of the world.  An agent that uses such a model is called a model- based agent. Model-based reflex agents  The interesting part is the function UPDATE-STATE, which is responsible for creating the new internal state description. The details of how models and states are represented vary widely depending on the type of environment and the particular technology used in the agent design.  the box labeled “what the world is like now” represents the agent’s “best guess”.  Uncertainty about the current state may be unavoidable, but the agent still has to make a decision. Goal-based agents  Knowing something about the current state of the environment is not always enough to decide what to do.  The agent needs some sort of goal information that describes situations that are desirable  The agent program can combine this with the model (the same information as was used in the model based reflex agent) to choose actions that achieve the goal.  Sometimes goal-based action selection is straightforward— for example, when goal satisfaction results immediately from a single action.  Sometimes it will be more tricky—for example, when the agent has to consider long sequences of twists and turns in order to find a way to achieve the goal.  Search and planning are the subfields of AI devoted to finding action sequences that achieve the agent’s goals  The goal-based agent’s behavior can easily be changed to go to a different destination, simply by specifying that destination as the goal Utility-based agents  Goals alone are not enough to generate high-quality behavior in most environments.  Goals just provide a crude binary distinction between “happy” and “unhappy” states.  A more general performance measure should allow a comparison of different world states according to exactly how happy they would make the agent.  Because “happy” does not sound very scientific, economists and computer scientists use the term utility instead.  UTILITY FUNCTION  An agent’s utility function is essentially an internalization of the performance measure. If the internal utility function and the external performance measure are in agreement, then an agent that chooses actions to maximize its utility will be rational according to the external performance measure.  Like goal-based agents, a utility-based agent has many advantages in terms of flexibility and learning.  A model-based, utility-based agent. It uses a model of the world, along with a utility function that measures its preferences among states of the world. Then it chooses the action that leads to the best expected utility, where expected utility is computed by averaging overall possible outcome states, weighted by the probability of the outcome. Learning agents  A learning agent can be divided into four conceptual components, as shown in Figure 2.15.  The most important distinction is between the learning element, which is responsible for making improvements, and the performance element, which is responsible for selecting external actions.  The performance element is what we have previously considered to be the entire agent: it takes in percepts and decides on actions.  The learning element uses feedback from the critic on how the agent is doing and determines how the performance element should be modified to do better in the future.  The last component of the learning agent is the problem generator. It is responsible for suggesting actions that will lead to new and informative experiences.  How the components of agent programs work  Roughly speaking, we can place the representations along an axis of increasing complexity and expressive power—atomic, factored, and structured. To illustrate these ideas, it helps to consider a particular agent component, such as the one that deals with “What my actions do.”  This component describes the changes that might occur in the environment as the result of taking an action, and Figure 2.16 provides schematic depictions of how those transitions might be represented.