Statistics and Bayesian Analysis Quiz

Statistics II - Exam Study Guide

• Basics of Probability: Probabilities form the basis for statistical inference. Inferential statistics are used to determine how representative data are of a population. Probability involves predicting outcomes, while statistics involves interpreting data to make inferences about the population.

Probability & Statistics

• Frequentists vs Bayesians: Frequentists define probability as long-run frequency. For example, a fair coin (50% probability of heads) is expected to land heads in half of the trials. Frequentists require data, models, and design for their analysis. Bayesians, on the other hand, view probabilities as degrees of belief held by a rational agent.

Advantages & Disadvantages

• Frequentist view: Objective, unambiguous, and grounded in the physical world. However, infinite sequences don't exist, and it has a limited scope regarding the analysis of events.

• Bayesian View: Assigns probabilities to events based on beliefs and assumptions of an intelligent agent. Can handle events that aren't easily quantified in the physical world. However, it's subjective, requiring careful specification of belief. The Bayesian view is often considered too broad.

Probability Distributions

• Elementary events: For a given observation, the outcome will be one and only one of these events.
• Example: In tossing a coin, "heads" and "tails" are elementary events.

Statistics

• Binomial Distribution: This distribution applies when an event happens or doesn't happen (e.g., 0 or 1). Success probability (e.g., the probability of a 'heads' outcome in a coin toss) and the number of observations (trials) are important parameters defining this distribution.
• Example: Calculating probability of getting a specific number of successes (like getting heads 4 times in 20 coin tosses).

Relationship Between Models & Data

• Data = Model + Error
• Statistical Inference compares models to data

Using Different Distributions in R

• Binomial Distribution (dbinom, pbinom, rbinom, qbinom): Used to calculate outcomes and probabilities in experiments of finite sizes (e.g., number of heads in a series of coin flips)
• Example: calculating the probability of getting 4 heads in 10 coin flips.
• Normal Distribution (dnorm, pnorm, rnorm, qnorm): Used to calculate outcomes and probabilities when dealing with continuous data or distributions approximated by it.

Characteristics of Normal Distribution

• The area under the normal distribution curve is equal to 1.
• The mean, mode, and median are all equal in a normal distribution.
• The curve is symmetric around the mean (μ).
• Standard deviation (σ) controls the spread, which tells us if the data is closely clustered around the mean, or spread out.

Binomial vs Normal

• Binomial: Discrete (countable) plot appearance.
• Normal: Continuous (uncountable) smooth curve distribution

Functions in R for Correlation

• cor(), cor.test(), and rcorr() for calculating Pearson and Spearman correlations.

Sample Statistics and Population Parameters

• Population parameter: Describes the characteristic of the whole population.
• Sample statistic: Describes the characteristic of a smaller group (subset) taken from the population.

Running & Interpreting R Output

• Linear Regression: A method to find the relationship between variables. Results (output) often include estimates, standard errors, p-values, and R-squared values.

Hypotheses & Research Questions

• Hypothesis: A statement about the relationship between variables. Example: There is a relationship between the amount of exercise and overall health. Hypotheses testing attempts to rule out chance as a plausible explanation for results.

Effect Sizes

• Cohen's d: Effect size measure focusing on mean differences in terms of the standard deviation, mostly tested with student's t-tests or z-tests.

Sampling Theory

• Population: A comprehensive set of units to which findings are generalized.
• Sample: A subset of the population from which inferences about the population are drawn.
• Sampling distribution: A probability distribution of a statistic calculated to determine the distribution of outcomes for a given statistic in a population.

Correlation & Covariance

• Correlation: Measures the extent to which two variables are related.
• Covariance: Indicates how much two variables change together. Positive covariance signifies that they generally change in the same direction, while negative covariance means they change in opposite directions.

Partial and Semi-Partial Correlation

• Partial correlation: Measures the relationship between two variables while controlling the effect of other variables.
• Semi-partial correlation: Controls for a variable's effect on only one outcome variable (either X or Y).

Regression/Test Statistics

• Regression line: Straight line depicting the mathematical relationship between variables.

Ordinary Least Square

• Method of calculating regression line minimizing the difference between observed and predicted data.

Testing the Model (ANOVA)

• ANOVA (Analysis of variance): Test showing whether the variation of the model explains the variability in data better than the variation from the mean.

Mean Squared Error (MSE)

• MSE: Represents the variability in a distribution, often estimated using a model.
• MSE can be divided into Sum of squares of the model (SSM) and Sum of squares of the residuals (SSR).
• The proportion of variance explained by a model is often quantified as R-squared (R²).

Standard Error

• Standard error: A measure of the variability of a statistic.

Null and Alternative hypotheses

• Null hypothesis(H₀): A claim of "no difference" in a population.
• Alternative hypothesis(Ha): Contends that the null hypothesis is false.

Types of Sampling Methods

• Random Sampling: Every member has an equal chance of being selected.
• Stratified Sampling: The population is divided into subgroups (strata), and random samples are taken from each subgroup.
• Volunteer Sampling: Participants self-select to participate in a study.
• Opportunity Sampling: Choosing participants who are readily available.
• Convenience Sampling: Choosing participants that are convenient for the researcher.
• Snowball Sampling: Participants recruit other potential participants (useful for hard-to-reach populations).

Confidence Intervals

• Confidence intervals: Provide a range of plausible values for a population parameter (e.g., mean, proportion), based on a random sample. Example: There's a 95% chance that the average IQ is between 89 and 111.

Central Limit Theorem

• Central Limit Theorem: Shows that as sample sizes grow, a distribution of sample means gets closer and closer to a normal distribution.

Type I and Type II Errors

• Type I error: Rejecting a true null hypothesis (false positive)
• Type II error: Failing to reject a false null hypothesis (false negative)

Interaction Terms in Multiple Regression

• Interaction: When the relationship between two predictors differs depending on the level of a third variable. Example: The effect of one ingredient in a cake differs depending on the amount of another ingredient added to it.

Categorical Variable Coding

• Dummy coding: Used to represent categorical variables in an analysis.
• Unweighted effect coding: Assign values based in the set of groups and means.
• Weighted effect coding: An approach to coding categorical variables where the values are assigned based on a weight.
• Contrast coding: Useful in situations where the researcher has pre-existing hypotheses about the interactions of the variables.

Growth Models with Polynomials

• Example: Determining functional relationship between weight and time. This is done using interaction terms when examining more than one predictor.

Running & Interpreting R output

• Polynomial Regression: A technique for modelling curvilinear relationships.

Multiple Linear Regression

• Method to model the relationship between a dependent variable and two or more independent variables.