Understanding Probability

• Probability indicates how likely an event is to occur

Basic Concepts

• An experiment is a process that yields an observable outcome
• Sample space encompasses all potential outcomes of an experiment
• An event is a sample space subset, representing a specific result
• Mutually exclusive events cannot happen simultaneously
• Independent events are unaffected by each other's occurrence

Defining Probability

• Classical probability assumes equally likely outcomes, dividing favorable outcomes by total possible outcomes
• Empirical probability is based on observations, calculated as the number of event occurrences divided by total observations
• Subjective probability relies on personal judgment to assess the likelihood of an event

Basic Rules of Probability

• Event probabilities range from 0 to 1, inclusive
• The sum of all probabilities in a sample space equals 1
• The complement rule states the probability of an event not happening is 1 less the probability of it happening
• The addition rule says the probability of either of two mutually exclusive events occurring is the sum of their individual probabilities
• The multiplication rule says the probability of two independent events both occurring equals the product of their probabilities

Conditional Probability

• It is the likelihood of event A occurring given event B has already occurred
• It is denoted as P(A|B)
• It is calculated as P(A|B) = P(A and B) / P(B), provided P(B) > 0

Bayes' Theorem

• It updates the probability of a hypothesis based on new data
• Formula: P(A|B) = [P(B|A) * P(A)] / P(B)
• P(A|B) represents the posterior probability of A given B
• P(B|A) represents the likelihood of B given A
• P(A) represents the prior probability of A
• P(B) represents the prior probability of B

Discrete Probability Distributions

• Describes the probability for each value of a discrete random variable
• A discrete random variable has a finite or countably infinite number of values
• Probability Mass Function (PMF) gives the probability a discrete random variable equals a specific value

Common Discrete Distributions

• Bernoulli distribution describes the probability of success or failure in a single trial
• Binomial distribution counts successes in a set number of independent Bernoulli trials
• Poisson distribution counts events within a fixed interval

Continuous Probability Distributions

• A continuous random variable's probability of falling within a range of values
• A continuous random variable can take any value within a given range
• Probability Density Function (PDF) describes the likelihood of a continuous random variable taking a specific value

Common Continuous Distributions

• Uniform distribution gives equal likelihood to all values within a range
• Exponential distribution models the time until an event
• Normal distribution is a symmetric, bell-shaped distribution defined by mean and standard deviation
• Standard Normal Distribution follows a normal distribution with a mean of 0 and a standard deviation of 1

Joint Probability

• It's the likelihood of two or more events happening together
• It is denoted as P(A and B) or P(A, B)
• For independent events, P(A and B) = P(A) * P(B)

Marginal Probability

• The probability of a single event regardless of other events
• It is derived from the joint probability distribution by summing or integrating over other variables

Covariance

• Measures how two random variables change in tandem
• Positive covariance means variables increase or decrease together
• Negative covariance means one variable increases as the other decreases
• Zero covariance means variables are uncorrelated

Correlation

• It's a standardized measure of the linear relationship between two random variables
• Ranges from -1 to +1
• +1 indicates a perfect positive linear relationship
• -1 indicates a perfect negative linear relationship
• 0 indicates no linear relationship

Expected Value

• The average value of a random variable over the long run
• For discrete random variables, it is a sum of each value times its probability
• For continuous random variables, it is the integral of the variable times its PDF

Variance and Standard Deviation

• Variance measures the spread of a random variable around its mean
• Standard deviation is the square root of variance, measuring spread in the same units as the variable

Applications of Probability

• Risk assessment and management utilize probability concepts
• Statistical inference and hypothesis testing rely on probability
• Machine learning and data analysis use probability models
• Probability models uncertain events in various fields
• It is used for Decision making under uncertainty

Combinations and Permutations

• Combinations select items where order doesn't matter
• Permutations arrange items where order matters
• These are used to count arrangements or selections

Set Theory in Probability

• Probability uses set theory for defining events and relationships
• Union: Either A or B or both occur (A ∪ B)
• Intersection: Both A and B occur (A ∩ B)
• Complement: A does not occur (A')

Probability Distributions and Simulations

• Used in simulations to understand system behavior
• Monte Carlo simulations use random sampling for numerical results, where analytical solutions are hard to get