Statistics: Introduction

Statistics: Introduction

• Statistics involves the collection, analysis, and presentation of data
• Descriptive statistics aims to describe and summarize a dataset without inferring about a larger population.
• Inferential statistics allows us to make inferences about a population based on data from a sample.
• Key components of descriptive statistics include measures of central tendency, measures of dispersion, frequency tables, and charts.
• Measures of central tendency, such as mean, median, and mode, represent the central value of a dataset.
• Measures of dispersion describe how spread out the values in a dataset are, including variance, standard deviation, range, and interquartile range.
• Frequency tables show the frequency of each distinct value in a dataset.
• Contingency tables (cross-tabs) analyze relationships between two categorical variables, displaying the number of observations in each category combination.
• Charts and graphs visually represent data, including bar charts, pie charts, histograms, box plots, violin plots, and rainbow plots.

Hypothesis Testing: T-test

• T-tests analyze if there's a significant difference between the means of two groups.
• Types include one-sample, independent samples, and paired samples.
• One-sample compares a sample mean to a known reference mean.
• Independent samples compare means of two independent groups.
• Paired samples compare means of two dependent groups (paired measurements).
• T-test assumptions: metric data, normal distribution, and equal variances (for independent samples).
• The null hypothesis assumes no difference; the alternative hypothesis claims a difference.
• The T-value is calculated using the difference between means and standard error.
• The P-value represents the probability of observing a sample as extreme (or more extreme) than the observed sample, assuming the null hypothesis is true.
• A statistically significant result occurs when the P-value is less than the significance level (often 0.05), suggesting the observed difference is unlikely due to chance.
• Type I error: rejecting a true null hypothesis.
• Type II error: failing to reject a false null hypothesis.

Hypothesis Testing: ANOVA

• ANOVA (Analysis of Variance) tests for statistically significant differences between the means of three or more groups.
• One-way ANOVA examines differences based on one independent variable.
• Null hypothesis: all group means are equal; alternative hypothesis: at least one group mean is different.
• Key assumptions: metric dependent variable, independent observations, normal distribution within each group, and equal variances across groups.
• The F-statistic is the ratio of between-group variance to within-group variance.
• The P-value indicates the probability of an extreme F-statistic, assuming the null hypothesis is true.
• If the P-value is less than the significance level, reject the null hypothesis, indicating a significant difference between group means.
• Post hoc tests follow significant ANOVA results to determine which specific groups differ.

Hypothesis Testing: Two-Way ANOVA

• Two-way ANOVA explores the effects of two categorical independent variables (factors) on a continuous dependent variable.
• Examines main effects of each factor and the interaction effect between them.
• Assumptions similar to one-way ANOVA: normality, homogeneity of variances, and independence of observations.

Hypothesis Testing: Repeated Measures ANOVA

• Repeated measures ANOVA tests significant differences between means of three or more dependent samples (same participants measured multiple times).
• Null hypothesis: no differences between condition means; alternative hypothesis: condition means differ.
• Assumptions: normal distribution of dependent variable, sphericity (equal variances of differences between factor levels/time points).
• F-statistic and P-value calculations are similar to other ANOVA types.
• Post hoc tests identify specific differences among groups.

Hypothesis Testing: Mixed Model ANOVA

• Mixed model ANOVA combines between-subjects and within-subjects factors in one analysis.
• Between-subjects: different subjects assigned to levels of a factor.
• Within-subjects: same subjects exposed to all levels of a factor.
• Examines main effects and interaction effects.
• Assumptions: normality, homogeneity of variances (between-subjects and within-subjects), homogeneity of covariances (sphericity for within-subjects), and independence of observations.

Parametric vs. Nonparametric Tests

• Parametric tests have greater power but require assumptions (e.g., normality), while nonparametric tests make fewer assumptions, using data ranks instead of raw values.
• Nonparametric counterparts for parametric tests:
  • Mann-Whitney U (independent samples T-test)
  • Wilcoxon signed-rank (paired samples T-test)
  • Kruskal-Wallis (one-way ANOVA)
  • Friedman (repeated measures ANOVA)

Checking for Normal Distribution

• Data normality is essential for using parametric tests.
• Checked analytically (Kolmogorov-Smirnov, Shapiro-Wilk, Anderson-Darling) or graphically (histograms, QQ plots).
• P-values from tests indicate if the null hypothesis of normality should be rejected or retained.
• QQ plots compare data quantiles to theoretical normal quantiles. Departures from a straight line suggest non-normality.

Testing for Equal Variances

• Levene's test assesses equality of variances across groups, used with T-tests and ANOVAs.
• If P-value > 0.05, the assumption of equal variances is not rejected.

Correlation Analysis

• Correlation analysis measures the strength and direction of a linear relationship between two variables.
• Correlation coefficients range from -1 to +1. +1 = strong positive linear relationship; -1= strong negative; 0 = no linear relationship.

Pearson Correlation

• Measures linear relationship between two metric variables.
• Formula involves covariance and standard deviations.
• Can be tested for significance (P-value).
• Assumptions: metric data and normal distribution for both variables (if testing for significance).

Spearman Rank Correlation

• Nonparametric measure of association for ordinal variables or metric variables with unknown distribution.
• Uses ranks instead of raw values.
• Formula similar to Pearson but applied to ranks.

Kendall's Tau

• Another nonparametric measure of association for ordinal variables.
• Less sensitive to outliers than Pearson.
• Calculated using concordant and discordant pairs.
• Suitable for datasets with few values and many rank ties.

Point-Biserial Correlation

• Pearson correlation variant where one variable is dichotomous (two levels) and the other is metric.
• Calculates the means of the metric variable for each group of the dichotomous variable.
• P-value indicates the statistical significance of the observed correlation (relationship between variables).

Understanding Causality

• Correlation does not imply causation.
• To establish causation, need significant correlation, chronological sequence, controlled experiments, and a plausible theory explaining the influence of variables.

Regression Analysis

• Regression models the relationship between variables to predict a dependent variable based on independent variables.
• Simple linear regression uses one independent variable.
• Multiple linear regression uses two or more independent variables.
• Logistic regression predicts categorical outcomes (especially binary).

Simple Linear Regression

• Equation: y = a + bX, where y is the dependent variable, X is the independent variable, a is the y-intercept, and b is the slope.
• Slope indicates dependent variable change per unit change in independent variable.
• Y-intercept is the predicted value of y when X=0.
• Assumptions: linear relationship, independent errors, homoscedasticity (equal error variance), and normally distributed errors.
• P-value assesses significant relationship between variables.

Multiple Linear Regression

• Equation: y = a + b1X1 + b2X2 + ... + bKXk
• Coefficients (b) represent the impact of each independent variable on the dependent variable.
• Intercept (a) is predicted y if all Xs are zero.
• Assumptions: linear relationship, independent errors, homoscedasticity, normally distributed errors, and no multicollinearity.
• Multicollinearity: high correlation between independent variables which hinders isolating individual effects. Detected by variance inflation factor (VIF) < 10 and tolerance > 0.1. Fix it by removing a variable or combining them.

Logistic Regression

• Predicts categorical (especially binary) outcomes.
• Formula based on the logistic function, transforming linear combinations into probability (0-1).
• Coefficients affect the outcome's likelihood.
• Odds ratio calculated from exponentiated coefficients, representing the change in odds for a one-unit increase in an independent variable.
• Assumptions: linear relationship between independent variables and the logit of the dependent variable, independent errors, and homoscedasticity.

Cluster Analysis: K-Means Clustering

• Unsupervised clustering method for grouping data points based on similarity
• Algorithm:
  • Define number of clusters (K).
  • Randomly initialize cluster centroids.
  • Assign each data point to its closest centroid.
  • Recalculate cluster centroids.
  • Repeat until cluster solution stabilizes.
• Elbow method used to determine the optimal number of clusters.

Confidence Intervals

• Provide a range within which the true population parameter likely falls.
• Interpretation: if many samples are taken, 95% of intervals constructed will contain the true value. A statement about the method's long-run reliability.

Notes on Frequentist Statistics and Bayesian Approach

• Frequentist: True parameter is fixed, unknown value; Bayesian: parameter is a random variable with its own probability distribution.
• Confidence interval (Frequentist); Credible interval (Bayesian).
• Critics of Bayesian: due to the influence of prior distributions, it may not be fully objective.