Quiz 1: All Lectures PDF

Quiz 1: All lectures Lecture 1 - Intelligent agents Agents and their environments - Autonmous - Interacting Why study software agents? We need a computational architecture intended to handle interactions between AI-based software and int’s environment Defintions Agent Anything that can be...

Quiz 1: All lectures Lecture 1 - Intelligent agents Agents and their environments - Autonmous - Interacting Why study software agents? We need a computational architecture intended to handle interactions between AI-based software and int’s environment Defintions Agent Anything that can be viewed as perceiving its environment through sensors ans acting upon that environment through actuators. Robotic agent Sensors: RGB/infrared cameras, etc. Actuators: Motors, arms, etc. Software agent Sensory inputs: File contents, Network packets, Human input (keyboard/mouse/touchscreen) Outputs Actautors: Information displayed, sound generation etc. An agents choice of action at any given instant can depend on its built-in-knowledge Input => Agent Function => Output Ex: Vacuum-cleaner world Sensory perceptions (locations): A and B Sensory perceptions (status): Dirty or Clean Actions: (go to) Right, (go to) Left, Such or Do Nothing The vacuum cleaners start in A Main goal of the agent: Clean A and B in an optimal way. Percept-action sequences Can make a table of the possible percept-action sequences (all possible decisions) Agent function function REFLEX-VACUUM AGENTS([location, status]) returns an action if status = Dirty then return Suck etc… Reacting to the input! (Basic idea) Rational agents => Agents doing things “right” Performance measure = Evaluates any given sequence of environment states. In humans, desirable actions form our point of view In software agents, is the mind of the designer Percept sequence Action Is it desirable? Better to design a performance measure according to what one actually wants to be achieved in the environment, rather than according to how one thinks the agent should behave. => Rational decisions Definition: Rational agent For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in-knowledge the agent has. PEAS => necessary to describe the problem Performance Environment Actuators Sensors Simple reflex agents These agents select actions on the basis of the current percept, ignoring the rest of the percept history. Ex: Reflex-Vacuum-Agent Condition-action rule: If then Pros - Computationally simple - Fast during decision-making - Works only if the environment is fully-observable Cons - A little bit of unobservability can cause serious trouble - No memory The firefighter agent The agent architecture (functions in the code) is based on the BEN (Behaviour with Emotions and Norms) architecture Input: Environment Output: Next movement Perceptions: - Location - Water level Model-based reflex agents Fixing problems of the simple reflex agents. - For handling partial observability: Keep track of the part of the world it can’t see now using an internal state Performance = What my actions do Access information if the previous action was successful or not. Updating the internal state requires: - Information about how the world involves independently of the agent. - Information about the effects of the agent’s actions. => Definition: Transition model of the agent’s world. Model-based agent is the model of an agent that uses a transition model + sensor model. Goal-based agents Need some sort of goal information that describes situations that are desirable. Keeps track of world and have a set of goals, chooses the action that will eventually lead to the achievement of its goal. Firefighter agent Desires/intention = Goals = Desires Utility-based agents A utility function could be used for solving when there are conflicting or uncertain goals, and a decision needs to be made. Solve scenarios with partial observability A rational utility-based agent chooses the action that maximizes the expected utility of the action outcomes. It requires the utility of each outcome! Learning agents Performance element box = Represent what we have previously considered to be the whole agent program (e.g, reflex agents) Critic = Assesses how the agent is doing, considering an external performance standard. Necessary because the percepts themselves provide no indication of the agent’s success Learning element = Gets to modify that program to improve its performance. Lecture 2 - Knowledge-based agents, cognitive architectures and multi-agents Knowledge-based agents An agent with a knowledge base - Suppose that an agens has a repository of knowledge, so-called knowledge base (KB) Depending on the world's status and the KB's previous knowledge, we can ask the KB what action to take.  Tell Almost like a relational database, but it’s not. Definition: Knowledge-based agent: Use a process of reasoning over an internal representation of knowledge to decide what action to take. It considers a knowledge sentence step at a specific time (t) t It flies If it’s a penguin it doesn’t fly If it’s a penguin => It’s a bird If it’s an eagle => It’s a bird fly(x) :- bird(X) …. (A rule) fly(X) beta, alfa) / beta Entailment: alfa entails the sentence beta, if alfa is true beta is true Cognitive architecture in software agents How humans act? Philosophy of mind => Practical Reasoning => The standard conception of action provides a conception of agency. A being has the capacity to exercise agency just in case it has the capacity to act intentionally The exercise of agency consists in the performance on intentional actions, and in many cases in the performance of unindented actions Belief Desire Intention (BDI) model Belief Definition Represent an agent’s information about the world. They are facts and knowledge tjat the agent holds to be true. Role: Beliefs help the agent understand the current state of the worl and predict the outcome of different actions. Desires Definition: Represent the goal or objective. Role: Desires provide the direction for the agent’s actions. Intentions Definition: Commitments that the agent makes to specific plans of action. Chosen means to achieve desired goal. Role: Intentions guide the agent’s actions by providing a plan to follow. Key steps Inputs: Initial beliefs and intentions. wish() function that generate desires (goals) focus() prioritize functions based on current BDIs plan() set of actions based on updated intentions while alive do p Functionf => Output f(x) = y Some notation  ’ Supervised ML Different steps  1. The training of the model 2. The use of the model For the training, use ECG data point as input and the labeling set is called the output. During training Probabilistic models are able to represent uncertainty in the model's predictions. Definition: Methods for learning a predictive model f(x) = y of the relationship between: - features of possible relevance x (input/feature) - outcome of interest y (output/label/target) from observed training data Each data point in the training data provides a snapshot of how y depends on x, and the goal in supervised ML is to squeeze as much info as possible out of the training set. Notation: It is used a vector notation x to denote the input since it is assumed to be a p-dimensional vector: ^T denotes the transpose. The input x is a high-dimensional vector. For example, in pixels of a picture, the input is a greyscale image, and x can be all pixel values in the image: p = h x w The output y, is often low dimension. The type of output value could be numerical or categorical, which is important because it is possible to distinguish the type of ML problem: regression and classification. The goal of Supervised ML Making predictions for new previously unseen test inputs For example, given three goal shots (angles-distances) predict the probability of a goal in a new position. Types of supervised learning data Numerical variables take on numerical values, e.g social media engagement (y = “watch time of YouTube videos”) Categorical variables take on values in one of K distinct classes, e.g Spam filter Classification = When target y is categorical Regression = When target y is numerical Regression Input variable x Output variable y Learning a model predicting y for a given x, when y is numerical. The model assumption is y = f(x; beta) + epsilon beta = parameters of the model epsilon = noise Linear regression: f is a linear model, we have y = beta0 + beta1x Linear Regression model Assumes that the output variable y (a scalar) can be described as an affine combination of the p input variables, x1, x2, ….., xp plus a noise term epsilon. We will use the vectorized version of the parameters Parametric version of the linear regression model: We want to minimize the error (distances) E! Loss functions and Cost functions How do we know that the model is “accurate”? Loss function = Measures how close the model's prediction is to the observed data y. Cost function = Average loss over the training data Training a model then amounts to finding the parameter values that minimize the cost. Examples of loss functions Could be whatever mathematical function for optimizing Least Square  - The goal is to choose the model (blue line) so that the sum of the squares (light red) of each error E is minimized. Polynomial Regression Allows you to fit a curved line to your data, which can be more flexible and accurate than a straight line when the relationship between variables is complex. Example: Overfitting The model fits too precisely to the training data. Can not generalize well. How to overcome it? Using regularization. Classification Learns a model that can predict a class y for an input x A classification model can be specified in terms of the conditional class probabilities The probability of class m given that we know the input x. A binary classification task has M = 2, thus y = 1 or y = 0. We need a mathematical function that help us predict probabilities for binary outcomes (true/false, 0/1) See below Logistic function Known as the sigmoid function, and gives rise to logistic regression. Gives 0 or 1, which can be interpreted as probabilities. Distinguish between two classes. Decision boundary By setting a threshold (usually 0.5) we can easily classify instances. Ex: Predict if an email is spam or not. Logistic function will take features of the email (number of certain keywords) and output a probability. Logistic Regression Model for binary classification Viewed as a modification of the linear regression model so that it fits the classification (instead of regression) problem. The input space can be segmented into M regions, separated by decision boundary. Where do we put the line? Same as in linear regression, what line fits the points, but now what line splits the classes the best? Support vector machine To find the optimal boundary (or hyperplane) that best separates different classes in the dataset. Support Vectors are the data points that are closest to the hyperplane. They are critical because they define the position and orientation of the hyperplane. We want a classifier with as big margin as possible. The goal of the Support vector machine is to maximize this margin, making the classifier more robust and less likely to overfit. k-Nearest Neighbors (k-NN) In classification, the k-NN algorithm finds the k nearest neighbors in the training set. The class of the new datapoint is determined by the majority class among these neighbors. With a small value on k might result in overfitting. Decision boundaries are shown as the black lines. The choice of the tuning parameter k, controls the model flexibility Small k => Small bias, Large variance => Tend to overfit Large k => Large bias, Small variance => Tend to underfit Unsupervised ML Methods for learning a model of - Features of possible relevance x (input/feature) - No labels y From observed training data Goals Find a structure in data => Clustering Find distribution of the data => Density Estimation Change the representation => Dimensionality Reduction Clustering Dataset T = {x1,...., xn} Notice no labels! Distance function d: T x T => R Partition k-Means k-Means - Divide the data in k different clusters - FInd a representative for every cluster Function that minimize the step! 1. Measure the distance  2. Calculate new center Repeat until distance between previous center and new is minimum. k-Means issues - The categorization is hard - Problem with local optima - k must be defined at the beginning - k-Means wants to find round, convex clusters - k-Means is a discriminative model Problem when data looks like this, there is no circular center: Gaussian Mixture Model Expectation Maximization Algorithm: A powerful optimization technique used for estimating parameters in statistical models when there is missing or incomplete data such as the parameters of our Gaussian in the GMM model. The main idea of the Gaussian Mixture Model - Create probabilities of the clusters - Gives very nice shapes - Clusters in different shapes k-Means VS GMM k-Means - Deterministic - Discriminative - Probability of orange is high => Certain decision GMM - Probabilistic - Generative - Probability of blue is high but probability of x is low => Uncertain decision Use GMM model to create new points => Generative! - Take data and create another point that does not exist that is part of that cluster.