Bivariate Statistical Analysis

Bivariate Statistical Series

• Bivariate data is data on two variables, paired together, and it's a specific case of multivariate data.

Data Types

• Bivariate data scenarios include both variables being quantitative, both being qualitative, or one of each.
• Qualitative variables are categorical like gender or ethnicity.
• Analyzing qualitative bivariate data involves contingency tables, chi-square tests, and graphical representations.
• Examples of qualitative variables include gender (Male, Female) and degrees (Liberal Arts, Business Admin, Technology).
• Analyzing Qualitative/Quantitative focuses on stratified analysis or methods like ANOVA or t-tests to compare means across groups.
• Quantitative variables represent numerical measurements like height or income.
• Analyzing quantitative bivariate data uses correlation analysis and linear regression, with correlation coefficients and scatterplots to understand the relationship.

Contingency Tables

• A two-dimensional statistic is any application C defined from a finite set Ω to R².
• X(w) is the response of individual w to variable X, making X the first marginal statistic.
• Y(w) is the response of individual w to variable Y, making Y the second marginal statistic.
• The table has k rows and l columns, where nij represents the frequency of people presenting modality xi of X and modality yj of Y.
• ni. is the subtotal of the frequencies of individuals in the i-th row, showing modality xi of X.
• n.j is the subtotal of frequencies in the j-th column, presenting modality yj of Y.

Relative Frequencies

• fij = nij / n is the proportion of individuals presenting modality xi of X and yj of Y.
• fi. = ni. / n is the relative frequency of individuals presenting modality xi of X.
• f.j = n.j / n is the relative frequency of individuals presenting modality yi of Y.
• The sum of fi., the sum of f.j, and the sum of fij all equal 1.

Marginal Distributions

• Sums of frequencies or relative frequencies in rows define the marginal distribution of variable X called the first marginal distribution.
• Sums of frequencies or relative frequencies in columns define the marginal distribution of variable Y called the second marginal distribution.

Conditional Distribution

• The conditional distribution of modality yj of Y given modality xi of X is fY=yj|X=xi = nij / ni..
• The conditional distribution of modality xi of X given modality yj of Y is fX=xi|Y=yj = nij / n.j.
• fx=xi|Y=yj = fij / f.j = fy=yj|X=xi / fi..

Mean and Variance

• Means and variances are defined as in basic statistics.
• X = (1/n) * Σ(ni. * xi) for all i.
• Y = (1/n) * Σ(n.j * yj) for all j.
• V(X) = σx² = (1/n) * Σ(ni. * xi²) - X² for all i.
• V(Y) = σy² = (1/n) * Σ(n.j * yj²) - Y² for all j.

Conditional Mean

• The conditional average formulas are:
  • mx|Y=yj = Σ(fx=xi|Y=yj * xi) for all i.
  • my|X=xi = Σ(fY=yj|X=xi * yj) for all j.

Conditional Variance

• The conditional variance formulas are:
  • V(X|Y = yi) = Σ(fx=xi|Y=yj * xi²) - mx|Y=yi
  • V(Y|X = xi) = Σ(fY=yi|X=xj * yi²) - my|X=xi

Independence

• Independence means no relationship between two statistical variables.
• Variables A and B are independent if the distribution of one doesn't vary with changes in the other.
• Mathematically, independence is assessed using conditional distributions.
• Variables are independent if the conditional distribution of variable A given variable B is identical across all modalities of variable B and vice versa.
• X and Y are said to be independent if fij = fi. * f.j for all i and j.
• If X and Y are independent, fx=xi|Y=yj = fx=xi, and fY=yj|X=x₁ = fy=yi.

Relationships Between Statistical Series

• This section focuses on the relationships between variables, including tools and techniques to analyze and quantify these relationships.

Covariance and Correlation Coefficient

• Covariance measures how two variables vary together.
• Positive covariance means both variables increase together; negative means they have an inverse relationship.
• Cov(X, Y) = σxy = (1/n) * ΣΣ(nij * (xi - X) * (yj - Y)).
• n is the total number of observations.
• nij is the frequency of joint occurrence of xi and yj.
• X and Y are the means of variables X and Y.
• Cov(X, X) = σxx = V(X).
• The correlation coefficient normalizes covariance, ranging from -1 to 1, for easier interpretation.
• Definition: r(X, Y) = Cov(X, Y) / (σx * σy).
• σχ and σy are standard deviations of X and Y, respectively, while σxy is the covariance between them.
• The the coefficient is degree of linear dependence between two variables, X and Y.
  • -0.75 < r(X,Y) < 0.75 suggests a weak correlation.
  • |r(X,Y)| ≥ 0.75 indicates a strong correlation.
• Property: If X and Y are linearly independent, the correlation coefficient is zero.

Simple Linear Regression

• Linear regression models the relationship between a dependent variable and one or more independent variables.
• It quantifies linear association for prediction and inference.
• Regression coefficients show how changes in independent variables affect the outcome.
• Affine function: f(x) = ax + b.
  • a is the slope, and b is the y-intercept.
• Goal: quantitatively measure how well a line fits the data.
• Error, e, is the signed vertical distance from the line to a data point.
• Least-squares criterion: the line that best fits has the smallest sum of squared errors.
• Regression line: the line that best fits a set of data points according to the least-squares criterion.

Coefficient of Determination

• The coefficient of determination (R²) measures how well a statistical model predicts an outcome on a scale of 0 to 1.
• It's the proportion of variation in the dependent variable predicted by the statistical model, and it's simply r²(X, Y).
  • R² = 0 means the regression does not explain the observed relationship.
  • R² = 1 means the regression perfectly captures the variability in the data.
• The closer the coefficient of determination gets to 0, the more the data points scatter around the regression line.
• As the coefficient approaches 1, the data points converge tightly around the regression line.

Outliers and Influential Observations

• An outlier is an observation outside the data's overall pattern.
• In regression, an outlier is far from the regression line.
• Outliers can significantly affect regression analysis.
• An influential observation is not a recording error.

Non Linear Shapes

• When the scatterplot appears stretched out, with a non-linear shape like logarithmic, exponential etc., alternative fitting methods are used.

Exponential Model

• Exponential models suit situations where growth begins slowly and accelerates, or where decay begins rapidly.
• Y = eax+b. where a should be non negative.
• In order to determine a and b, a logarithmic transformation is applied to the model, and we define Z = ln(Y)

Introduction to Time Series

• Time series analysis helps understand how data evolves, uncovering patterns, trends, and cycles.
• Definition: time series data are numerical data, indexed sequentially by a timestamp, their order being extremely important.
• Time series analysis methods extract meaningful statistics and characteristics from the data, and model to predict values based on historical values. Main objectives: Study of the phenomena over time, making decisions and forecasting based on the observed values.

Time Series Components

• Time series data (Xt) is the result of trend, seasonal component and random components.
• Trend: Long term direction of the data being analysed.
• Seasonal Component: Pertains to phenomena occurring at regular intervals over time.
• Residual Component (or Noise): Represents irregular fluctuations, random variables that are not based on season or trends.
• Accidental phenomena (strikes, extreme weather events, or financial crises) can significantly impact the behavior of time series data.

Time Series Data Visualisation

• Time plots visualize the time series data.

Time Plots

• Plotted as lines connecting observed values in order to appreciate patterns over time.
• The graph has to be chronological.

Seasonal Time Plots

• Slices the time series to identify seasonal patterns.
• 12 slices if the time series recorded monthly