Bayesian Machine Learning Lecture PDF

CHAPTER 7 - BAYESIAN MACHINE LEARNING BAYESIAN MACHINE LEARNING q Quantification of uncertainty is where Bayesian Inference meets Machine Learning. q A general way of thinking about reasoning under uncertainty. q Bayesian influence allows you essentially to move from the context of more Classi...

CHAPTER 7 - BAYESIAN MACHINE LEARNING BAYESIAN MACHINE LEARNING q Quantification of uncertainty is where Bayesian Inference meets Machine Learning. q A general way of thinking about reasoning under uncertainty. q Bayesian influence allows you essentially to move from the context of more Classical AI where we did a lot of representation using Logic and move to a scenario where we do knowledge representation by setting up Probability Distributions and then inference becomes the computation of conditional probabilities rather than a logical deduction using a theorem. BAYESIAN MACHINE LEARNING q We replace the knowledge base by a probability distribution that represents our beliefs about the world. q We replace the task of logical inference with the task of computing conditional probabilities. Dr Wided Lejouad Chaari BAYES RULE 𝑷 𝑯. 𝑷(𝑬|𝑯) q 𝑷 𝑯|𝑬 = 𝑷(𝑬) q What is it saying? § P(H) = Probability a hypothesis is true (before any evidence) § P(E|H) = Probability of seeing the evidence if the hypothesis is true § P(E) = Probability of seeing the evidence § P(H|E) = Probability a hypothesis is true given some evidence Dr Wided Lejouad Chaari BAYES RULE – EXAMPLE 𝑷 𝑯. 𝑷(𝑬|𝑯) ¡ 𝑷 𝑯|𝑬 = 𝑷(𝑬) ¡ H (Hypothesis): "It will rain” ¡ E (Evidence): "There are dark clouds in the sky." ¡ P(H): "The chance of rain based on the weather forecast.” ¡ P(E|H): "If it's going to rain, how likely is it that you'd see dark clouds?” ¡ P(E): "What is the chance of seeing dark clouds, regardless of whether it rains?” ¡ P(H|E): "After seeing the dark clouds, how likely is it that it will rain?” EXAMPLE … q Steve is very shy and withdrawn, invariably helpful but with very little interest in people or in the world of reality. A meek and tidy soul, he has a need for order and structure, and a passion for detail. q Which of the following do you find more likely: ¡ « Steve is librarian », or « Steve is a Farmer » Dr Wided Lejouad Chaari EXAMPLE … Dr Wided Lejouad Chaari EXAMPLE … q Anyone who is asked this question is not expected to have perfect information on the actual statistics of farmers, librarians,and their personality traits. q Rather, we make an estimation. q Rationality is not about knowing facts, it’s about recognizing which facts are relevant. Dr Wided Lejouad Chaari EXAMPLE … q We may start by picturing a representative sample of farmers and librarians, say 200 farmers and 10 librarians. q When you hear the meek and tidy soul description, you may say that 40% of librarians would fit that description and that 10% of farmers would. q From your sample, you’d expect that about 4 librarians and 20 farmers fit the description. Dr Wided Lejouad Chaari EXAMPLE … Dr Wided Lejouad Chaari EXAMPLE … Dr Wided Lejouad Chaari EXAMPLE … Dr Wided Lejouad Chaari EXAMPLE … 𝟒 𝑷 𝑳𝒊𝒃𝒓𝒂𝒓𝒊𝒂𝒏 𝒈𝒊𝒗𝒆𝒏 𝑫𝒆𝒔𝒄𝒓𝒊𝒑𝒕𝒊𝒐𝒏 = 𝟒*𝟐𝟎 = 16.7% q So even if you think a librarian is 4 times as likely as a farmer to fit this description, that’s not enough to overcome the fact that there are way more farmers. Dr Wided Lejouad Chaari EXAMPLE … Dr Wided Lejouad Chaari EXAMPLE … q The Key idea underlying Bayes Theorem is that new evidence should not completely determine your beliefs, it should update prior beliefs. Dr Wided Lejouad Chaari EXAMPLE … EXAMPLE … q The way seeing evidence restricts the space of possibilities, and the ratio you need to consider after that. q The numbers you’d estimate would be a little bit different, but what matters is how to fit the numbers together to update a belief based on evidence. Dr Wided Lejouad Chaari EXAMPLE … Dr Wided Lejouad Chaari EXAMPLE … Dr Wided Lejouad Chaari MORE MATHEMATICALLY … The general situation where Bayes Theorem is relevant is: § when you have some hypothesis (Steve is a Librarian), § and you see some evidence (verbal Description of Steve), § and you need to know the probability that the hypothesis holds, given that the evidence is True Dr Wided Lejouad Chaari MORE MATHEMATICALLY … Dr Wided Lejouad Chaari OUR GOAL … Dr Wided Lejouad Chaari OUR GOAL … P(H) the ratio of librarians to farmers in the general population. This is known as the prior. Dr Wided Lejouad Chaari ’’Likelihood’’ Afer that, we need to consider the proportion of librarians that fit this description. The Probability we would see the evidence given that the hypothesis is TRUE. The vertical bar means that we’re considering the proportion of a limited part of the total space of Possibilities. Dr Wided Lejouad Chaari Similarly, we need to know how much of the other side of our space includes the evidence. The probability of seeing the evidence given that our hypothesis isn’t True. Dr Wided Lejouad Chaari Dr Wided Lejouad Chaari 210 x 1/21 x 0.4 Dr Wided Lejouad Chaari Dr Wided Lejouad Chaari Dr Wided Lejouad Chaari Dr Wided Lejouad Chaari More abstract representation purely in terms of probabilities Dr Wided Lejouad Chaari P(E) – The total probability of seeing the evidence. Dr Wided Lejouad Chaari In practice, to calculate P(E), you break it down into the case where the Hypothesis is True and the one where it isn’t. Dr Wided Lejouad Chaari Dr Wided Lejouad Chaari BAYESIAN MODEL q Programmers use it in building Artifical Intelligence, where you sometimes want to explicitly and numerically model a machine’s belief q Bayes Theorem can reframe how you think about thought itself. Dr Wided Lejouad Chaari