Understanding Probability Theory

• Probability theory quantifies uncertainty.
• It serves as a framework to analyze random events and predict likelihood.
• Probability ranges between 0 and 1, where 0 signifies impossibility, and 1 signifies certainty.

Basic Concepts

• Sample space includes all possible outcomes of a random experiment.
• An event is a sample space subset, representing a specific outcome or group of them.
• Probability measure is a function assigning a probability to each event, which fulfills non-negativity, additivity, and normalization axioms.

Types of Probability

• Classical probability assumes equally likely outcomes in the sample space.
• Empirical probability uses observed data from repeated trials of an experiment.
• Subjective probability is based on personal beliefs or judgments about event likelihood.

Axioms of Probability

• The probability of any event must be greater than or equal to zero.
• For mutually exclusive events, the probability of their union is the sum of individual probabilities.
• The probability of the entire sample space equals one.

Conditional Probability

• It measures the chance of event A given that event B has already happened.
• Denoted as P(A|B)
• Calculated as P(A|B) = P(A & B) / P(B), given that P(B) > 0.

Independence

• Events A and B are independent if one's occurrence doesn't affect the other's probability.
• Mathematically, events A and B are independent if P(A|B) = P(A), or P(B|A) = P(B).
• Alternatively, P(A & B) = P(A) * P(B).

Random Variables

• This is a variable whose value comes from a numerical result of random phenomena.
• Discrete random variables can only have a finite number of values.
• Continuous random variables can take any value within a range.

Probability Distributions

• This function describes the probability of each value that a random variable can take.
• Discrete probability distributions assign probabilities to each discrete value.
• Continuous probability distributions describe probabilities across a continuous range, displayed via a probability density function (PDF).

Discrete Probability Distributions

• Bernoulli distributions model the probability of success or failure in a single trial.
• Binomial distributions model the number of successes within a set number of independent trials.
• Poisson distributions model the number of events in a fixed time or space.

Continuous Probability Distributions

• Normal distributions are bell-shaped, defined by mean and standard deviation, and are common due to the central limit theorem.
• Exponential distributions model the time until an event.
• Uniform distributions mean all values in a range are equally likely.

Normal Distribution

• Symmetrical and bell-shaped, defined by two parameters: mean (μ) and standard deviation (σ)
• The mean determines the distribution center, and the standard deviation determines the spread.
• Many natural phenomena follow or can be approximated via a normal distribution.
• Standard Normal Distribution: A normal distribution that has a mean of 0 and a standard deviation of 1
• Any normal distribution can be transformed into a standard normal distribution by standardizing the values (calculating z-scores)

Expected Value

• The average value of a random variable, weighted by its probabilities
• The expected value for a discrete random variable is the sum of each value multiplied by its probability
• E(X) = Σ [x * P(x)]
• For a continuous random variable, the expected value is the integral of x multiplied by its probability density function (PDF)
• E(X) = ∫ [x * f(x) dx]

Variance and Standard Deviation

• Variance measures the spread of a random variable around its expected value
• Variance is calculated using the expected value of the squared difference between each value and the expected value: Var(X) = E[(X - E(X))^2]
• Standard deviation is the square root of the variance.
• Expresses the spread in the same units as the random variable, making it more interpretable.

Chebyshev's Inequality

• It provides a lower limit to the probability that a random variable is within a certain distance of its mean, regardless of distribution.
• P(|X - μ| ≥ kσ) ≤ 1/k^2, where X is the random variable, μ is the mean, σ is the standard deviation, and k is any positive number
• As an example, at least 75% of values will fall within 2 standard deviations of the mean (k=2)

Central Limit Theorem

• The sum (or average) of many independent and identically distributed (i.i.d.) random variables will be approximately normally distributed, regardless of the original distribution.
• It allows inferences about population parameters based on sample statistics, even without knowing the population's underlying distribution.

Applications of Probability Theory

• Risk assessment involves quantifying and managing the risks of various events or decisions.
• Statistical modeling uses models to describe and predict random phenomena.
• Machine learning uses algorithms to learn from data and predict future outcomes.
• Finance uses probability to price financial instruments and manage investment portfolios.
• Insurance applies it to calculate premiums and assess claim risks.
• Engineering integrates probability theory to design reliable systems and assess the probability of failure.