AI - Lecture 3.pdf

PEAS Agent Part-picking robot  Performance measure: Percentage of parts in correct bins  Environment: Conveyor belt with parts, bins  Actuators: Jointed arm and hand  Sensors: Camera, joint angle sensors  PEAS Agent Interactive English tutor  Performanc...

PEAS Agent Part-picking robot  Performance measure: Percentage of parts in correct bins  Environment: Conveyor belt with parts, bins  Actuators: Jointed arm and hand  Sensors: Camera, joint angle sensors  PEAS Agent Interactive English tutor  Performance measure: Maximize student's score on test  Environment: Set of students  Actuators: Screen display (exercises, suggestions, corrections)  Sensors: Keyboard  Environment types  Fully observable vs. Partially observable: if the agent’s sensors give it access to the complete state of the environment is fully observable.  Single agent vs. Multiagent: An agent operating by itself in an environment. (competitive or cooperative multiagent ). The agent- design problems in multiagent environment are: communication, randomized behavior.  Deterministic vs Stochastic: if the next state of the environment is completely determined by the current state and the action executed by the agent, then we say the environment is deterministic; otherwise, it is stochastic.  Environment types  Episodic vs. Sequential: In an episodic task environment, the agent’s experience is divided into atomic episodes. In each episodes the agent receives a percept and then performs a single action. Crucially, in the next episode does not depend on the actions taken in previous episodes. In sequential environments, the current decision could affect all future decisions.  Static vs. Dynamic: The environment is changed while an agent is deliberating, then we say the environment is dynamic for that agent; otherwise it is static. Static environments are easy to deal with because the agent need not keep looking at the world while it is deciding on an action, nor need to worry about the passage of time. Dynamic environments, are continuously asking the agent about it wants to do. (The environment is semidynamic if the environment itself does not change with the passage of time but the agent's performance score does)  Environment types  Discrete vs. Continuous: The discrete/continuous distinction applies to the state of the environment, to the way time is handled, and to the percepts and the actions of the agent.  Known vs. unknown: it is not refers to the environment itself but to the agent’s (or designer's) state of knowledge about the “laws of physics” of the environment.  Environment types  We can expected that, the hardest case is: partially observable, multiagent, stochastic, sequential, dynamic, continuous, and unknown.  Taxi driving is hard in all these senses, expect that for the most part the driver’s environment is known.  The Structure of Agent  The job of AI is to design an Agent Program that implements the agent function (the mapping from percepts to actions).  This program is assumed to run on some sort of computing device with physical sensors and actuators  We call this the architecture Agent = Architecture + Program.  Agent Program  The agent program implements the agent function.  The appropriate design of the agent program depends on the nature of the environment.  Agent Programs  No physical agent in this universe will have the space to store the table.  The designer would not have time to create the table.  No agent could ever learn all the right table entries from its experience.  Even if the environment is simple enough to yield a feasible table size, the designer still has no guidance about how to fill in the table entries.  TABLE-DRIVEN-AGENT does do what we want.  Agent Programs Function TABLE-DRIVEN-AGENT (percept) returns an action Persistent: percepts, a sequence, initially empty Table, a table of actions, indexed by percept sequences, initially fully specified append percept to the end percepts Action LOOKUP(percept, table) Return action The TABLE-DRIVEN-AGENT program is invoked for each new percept and returns an action each time. It retrains the complete percept sequence in memory  Agent types Four basic types in order of increasing generality:  Simple reflex agents  Model-based reflex agents  Goal-based agents  Utility-based agents.  Learning agents. All these can be turned into learning agents  Simple Reflex Agents  These agents select actions on the basis of the current percept, ignoring the rest of the percept history.  For example, the vacuum agent is a simple reflex agent, because its decision is based only on the current location and on whether that location contains dirt.  Simple reflex behaviors occur even in more complex environments.  We can use condition-action rule, if car-in-front-is-braking then initiate-braking. Simple reflex agents  Simple reflex agents function Reflex-Vacuum-Agent( [location,status]) returns an action if status = Dirty then return Suck. else if location = A then return Right. else if location = B then return Left.  Problems with simple reflex agents  The agent work only if the correct decision can be made on the basis of only the current percept-that is, only if the environment is fully observable.  Infinite loops are often unavoidable for simple reflex agents operating in partially observable environment.  Escape from infinite loops is possible if the agent can randomize its action.  Model-based reflex agents  The most effective way to handle partial observability is for the agent to keep track of the part of the world it can’t see now.  The agent should maintain some sort of internal state that depends on the percept history and thereby reflects at least some of the unobserved aspects of the current state.  Example: for the driving tasks of changing lanes, the agent needs to keep track of where the other cars are if it can’t see them all at once.  Model-based reflex agents  Updating this internal state information requires two kinds of knowledge to be encoded in the agent program.  First: we need some information about how the world evolves independently of the agent.  Second: we need some information about how the agent’s own actions affect the world. An agent that uses a model is called a model-based agent. Model-based reflex agents It shows how the current percept is combined with the old internal state to generate the updated description of the current state, based on the agent’s model of how the world work.  Model-based reflex agents Function : MODEL-BASED-REFLEX-AGENT (percept) returns an action Persistent: state, the agent’s current conception of the world state model, a description of how the next state depends on current state and action rules, a set of condition-action rules action, the most recent action, initially none State UPDATE-STATE(state, action, percept, model ) Rule RULE-MATCH(state, rules) Action rule.ACTION return action  Goal-based agents  As well as the current description, the agent needs some sort of goal information that describes situations that are desirable.  Example: the taxi can turn left, turn right or go straight on, the correct decision depends on where the taxi is trying to get to.  The goal-based action selection could be a straightforward (when the goal satisfaction results immediately from a single action) or a tricky (when the agent has to consider long sequence action in order to find a way to achieve the goal). Goal-based agents  Goal-based agents  The decision making in goal-based agent is different from the condition action rules. Example: the reflex agent brakes when it sees brake lights. A goal-based agent, it will slow down. Thus the only action that will achieve the goal of not hitting other cars is to brake.  Problem of goal-based agent  Although the goal agent appears less efficient, it is more flexible because the knowledge that supports its decisions is represented explicitly and can be modified.  Example: if it starts to rain, the agent can update its knowledge of how effectively its brakes will operate; this will automatically cause all of the relevant behaviors to be altered to suit the new conditions. For the reflex agent, we would have to rewrite many condition-action rules.  Utility-based agents  A more general performance measure should allow a comparison of different world states according to exactly how happy they would make the agent. Because “happy” does not sound very scientific, economists and computer scientists use the term utility instead.  example: many action sequences will get the taxi to its destination but some are quicker, safer, reliable, or cheaper than others.  An agent’s utility function is essentially an internalization of the performance measure. If the internal utility function and external performance measure are in agreement, then an agent that choose actions to maximize its utility will be rational according to the external performance measure.  Utility-based agents  A utility-based agent can make rational decisions. first: when there are conflicting goals, only some of which can be achieved, the utility function specifies the appropriate tradeoff. second: when there are several goals that the agent can aim for, none of which can be achieved with certainty, utility provides a way in which the likelihood of success can be weighed against the importance of the goals. The utility-based agent has to model and keep track of its environment, tasks that have involved a great deal of research on perception, representation, reasoning, and learning. Utility-based agents  Learning agents  All agents can improve their performance through learning.  Learning allows the agent to operate in initially unknown environments and to become more competent than its initial knowledge alone might allow.  The learning element uses Learning agents feedback from the critic on how the agent is doing and determines how the performance element should be modified to do better in the future. Selecting Making external improvem actions. ent It is responsible for suggesting actions that will lead to new and informative experience  Summary Agents interact with environments through actuators and sensors The agent function describes what the agent does in all circumstances The performance measure evaluates the environment sequence A perfectly rational agent maximizes expected performance Agent programs implement (some) agent functions PEAS descriptions definie task environnements Environments are categorized along several dimensions: observable? deterministic? episodic? static? discrete? single-agent? Several basic agent architectures exist: reflex, reflex with state, goal-based, utility-based.