Probability & Statistics Lecture Notes PDF

Probability & Statistics Sapan H Mankad  Outcomes Students are expected to gain Knowledge about real world applications of Probability and Statistics in Machine Learning and related fields. Benefits of converting the knowledge and skills into job oppo...

Probability & Statistics Sapan H Mankad  Outcomes Students are expected to gain Knowledge about real world applications of Probability and Statistics in Machine Learning and related fields. Benefits of converting the knowledge and skills into job opportunities and/or career as a life long researcher or professional. 2 At the end of the session, you will be able to… 3  Contents Introduction The Law of Large numbers Probability and Probability Distributions Statistics Random Numbers vs. Random Variables Entropy Application Domains 7 August 2024 4 Let’s start with an exercise… The Law of Large numbers 7 August 2024 5  Empirical Probability vs. Theoretical Probability 7 August 2024 6  Intuitively, the probability of an event a could be defined as: Where N(a) is the number that event a happens in n trials 7 August 2024 7  Joint Probability Joint probability is the probability of two events happening together. Joint probability can also be described as the probability of the intersection of two (or more) events. Dependent vs. Independent events You have 52 candidates for a committee. Four are persons aged 18 to 21. If you randomly select one person, and then (without replacing the first person’s name), randomly select a second person, what is the probability both people will be between 18 and 21 years old? 7 August 2024 8  Statistics is a science of data which involves – Collecting – Classifying – Summarizing – Analyzing – Organizing – Interpreting numerical information 7 August 2024 9  Types of Statistics Descriptive Statistics Inferential Statistics 7 August 2024 10  Probability Distributions ◆ Probability Density Functions ◆ Cumulative Distribution functions ◆ 5 number summary 7 August 2024 11  Examples Tossing a coin Rolling a dice 8-bit gray scale image and its pixel intensity distribution Softmax function in neural network 7 August 2024 12  Measures Measures of central tendency – Mean – Mode – Median Measures of dispersion – Standard Deviation – Variance 7 August 2024 13  Measures of Center 7 August 2024 14  Measures of Center 7 August 2024 15  Measures of Variability Variability in statistics refers to how much the data points in a dataset vary or differ from one another. It is a measure of how spread out or dispersed the data is around the central tendency, such as the mean or median. Range Interquartile Range (IQR) Standard Deviation Variance 7 August 2024 16  Measures of Variability A dataset with high variability will have a wider range of values, while a dataset with low variability will have values that are more tightly clustered around the central tendency. 7 August 2024 17  Range Range: The range is a measure of variability in statistics that tells us the difference between the highest and lowest values in a dataset. The range can give us an idea of the spread or dispersion of the data in a dataset, but it only considers the two extreme values and does not take into account the distribution of the other values. 7 August 2024 18  Interquartile Range (IQR) Interquartile range (IQR) is a measure of variability in statistics that describes the spread of the middle 50% of the data in a dataset. It is calculated as the difference between the upper quartile (Q3) and the lower quartile (Q1) of the dataset. 7 August 2024 19  Variance Variance is a measure of variability in statistics that describes how spread out the data in a dataset are around the mean. It is calculated as the average squared deviation of each data point from the mean. 7 August 2024 20  Standard Deviation Standard deviation is a measure of variability in statistics that describes how much the data in a dataset are spread out around the mean. It is calculated as the square root of the variance, which is the average of the squared deviations of each data point from the mean. 7 August 2024 21  Measures of Position Measures of position give us a way to see where a certain data point or value falls in a sample or distribution. Percentiles Quantiles Standard Scores (Link) 7 August 2024 22  5 Number Summary Representation of 5-number summary of a dataset – Min – Quartile 1 (Q1) – Median (Q2) – Q3 – Max 7 August 2024 23  Visualization A box plot can be useful to compare two different sample distributions of the same population. An illustration using R 7 August 2024 24 7 August 2024 25  Example 7 August 2024 26  Random Numbers 7 August 2024 27  Random Variables 7 August 2024 28  Types of Random Variables (R.V.) 1. Continuous Random Variable 2. Discrete Random Variable 7 August 2024 29  Types of Random Variables (R.V.) Suppose the fire department mandates that all fire fighters must weigh between 150 and 250 pounds. The weight of a fire fighter would be an example of a continuous variable; since a fire fighter's weight could take on any value between 150 and 250 pounds. Suppose we flip a coin twice and count the number of heads. The number of heads? 7 August 2024 30  Random Variable Distributions Cumulative Probability Distribution (CDF) Probability Density Function (PDF) 7 August 2024 31 7 August 2024 32  Uniform Distribution 7 August 2024 33  Gaussian (Normal) Distribution The most important formula m 7 August 2024 34  Acknowledgement Carl Fredrich Gauss 7 August 2024 35  Let’s have another exercise…. Which of the two statements contains no information? 1. Tomorrow is an off-day. 2. Tomorrow is a working day. 7 August 2024 36  Entropy Entropy is a measure of uncertainty. The entropy for the facts which do not provide any new information is zero. For n equally probable possible messages, if the probability p is 1/n, then the information conveyed by a message is –log2(p). E = -σ𝑛𝑖=1 𝑝𝑖 𝑙𝑜𝑔𝑝𝑖 7 August 2024 37  Acknowledgement 7 August 2024 38  More on Entropy If a given probability distribution is uniform, then its entropy is 1. – Tossing a fair coin – P(0.5,0.5) – E(P) = 1 More uniform the distribution is, the greater is its entropy. – P(0.67,0.33) = 0.92 – P(1,0) = 0 (a biased coin) 7 August 2024 39  Entropy Usage Classification using Decision Tree Information Theory 7 August 2024 40  Application Domains Machine Learning Sentiment Analysis Image Processing & Video Processing Computer Vision Information Theory Speech/Audio/Music Queuing Theory Processing Networks Biometric Cloud Computing Authentication Stock Market Recommender Prediction Systems 7 August 2024 41  Bayes Theorem Next class 7 August 2024 42  Books A First Course on Probability by Sheldon Ross 7 August 2024 43  Disclaimer The contents of this presentation are compiled from various books and online resources available freely on Internet, and is to be used only for educational purpose. 7 August 2024 44