Probability & Statistics in Machine Learning

Introduction to Probability and Statistics

• Knowledge of Probability and Statistics is essential for applications in Machine Learning and various fields.
• Understanding these concepts can lead to job opportunities and research careers.

Key Concepts

• The Law of Large Numbers ensures that as more trials are conducted, the empirical probability will approach the theoretical probability.
• Empirical Probability is based on observed data, while Theoretical Probability is based on mathematical models.

Joint Probability

• Joint Probability refers to the occurrence of two events together, indicating the intersection of events.
• Example: Calculating the probability that two selected candidates are aged 18-21 without replacement.

Statistics Overview

• Statistics involves collecting, classifying, summarizing, analyzing, organizing, and interpreting data.
• Two main types of statistics are Descriptive and Inferential.

Probability Distributions

• Types of Probability Distributions include Probability Density Functions (PDFs) and Cumulative Distribution Functions (CDFs).
• The Five Number Summary includes Min, Q1, Median (Q2), Q3, and Max, providing insights into dataset distribution.

Measures of Central Tendency

• Central tendency is measured through Mean, Mode, and Median, which define the average or typical values in a dataset.

Measures of Dispersion

• Dispersion is analyzed using Standard Deviation and Variance, which show how data points spread around the central tendency.
• High variability indicates widely spread data, whereas low variability suggests data is closely clustered around the mean.

Range and Interquartile Range (IQR)

• Range is the difference between the highest and lowest values in a dataset, offering a crude measure of dispersion.
• Interquartile Range (IQR) measures the spread of the middle 50% of data, calculated as Q3 - Q1.

Variance and Standard Deviation

• Variance quantifies how much data points deviate from the mean, calculated as the average of squared deviations.
• Standard Deviation is the square root of the variance, indicating data spread in the same units as the original data.

Measures of Position

• Percentiles, Quantiles, and Standard Scores provide insights into the relative position of data points in a distribution.

Types of Random Variables

• Random Variables can be Continuous (e.g., weights within a range) or Discrete (e.g., counts of coin flips).

Random Variable Distributions

• Cumulative Probability Distribution (CDF) and Probability Density Function (PDF) explain random variable behavior.

Uniform and Gaussian Distributions

• Uniform Distribution has equal probabilities across all outcomes.
• Gaussian (Normal) Distribution is crucial for probabilities and statistics, characterized by its bell-shaped curve.

Entropy

• Entropy measures uncertainty, with zero entropy indicating no new information.
• For n equally probable messages, information conveyed = -log2(p), emphasizing distribution uniformity increases entropy.

Application Domains

• Understanding Probability and Statistics enhances analytical skills applicable across diverse domains, promoting clarity in data interpretation and decision-making.