Understanding Covariance

• Statistical work often assesses relationships between variables within sample data.
• Understanding how changes in one variable relate to changes in another is useful.
• Covariance and correlation are metrics to measure the association between two variables.

Covariance

• Covariance measures how two random variables vary together.
• Focus is on numeric variables.

Example: Age and Height

• For adolescents, as age increases, height also increases, showing a positive covariance.
• Among adults (ages 18-65), there is almost no covariance between age and height.
• Among elders (65+), as age increases, height decreases, showing a negative covariance.

Measuring Covariance

• Covariance of X and Y is expressed as cov(X, Y).
• Given n individuals, with X and Y measured for each, the covariance formula is provided.
• The formula calculates (xᵢ - x̄) * (yᵢ - ȳ) for each participant, indicating how far each participant's X and Y values are from their respective means.

Positive Covariance

• Positive covariance captures when two variables increase together (greater values of X are associated with greater values of Y, and vice versa).
• Represented as cov(X, Y) > 0.

Covariance Calculation Example

• With the help of an example with age and height, it is shown that multiplying two negative values (lower values of X and Y relative to their means) yields a positive value, consistent with positive covariance.
• The equation captures that above-average age tends to correlate with above-average height, and below-average age with below-average height.
• Focus is primarily on whether cov(X, Y) is negative, close to zero, or positive.

Negative Covariance

• Negative covariance occurs when higher values of X are associated with lower values of Y, and vice versa.
• Represented as cov(X, Y) < 0.

Covariance as Association Measure

• Covariance measures the association of two variables.
• It does not explain why variables vary together, only that they do.

Measuring and Visualizing Covariance in R

• Covariance can be calculated using the cov() function in R.
• The mtcars data frame in R has variables like miles per gallon (mpg) and horsepower (hp).
• High horsepower cars are expected to have fewer miles per gallon.
• Scatterplots can be used to visualize covariance.

Scatterplots with ggplot2

• The ggplot2 package is useful for generating visualizations in R.
• A basic scatter plot of miles per gallon (X) and horsepower (Y) can be created using ggplot() and geom_point().
• High horsepower values tend to have lower miles per gallon and can be observed visually.

Enhancing Scatterplots

• Additional features of ggplot2 can improve plot aesthetics, labels for name of the X-axis and Y-axis can be added, and overall titles to the graphic.
• The theme_minimal() function changes the appearance.

Interpreting Visual Patterns

• Positive covariance is seen when data "travels" from the lower left to upper right on a scatterplot.
• Negative covariance is seen when data "travels" from the upper left to the lower right.
• Weak covariance lacks a clear pattern, indicating that the variables behave independently.

Covariance Matrix Creation

• A covariance matrix is a two-dimensional array, with each row/column representing a variable.
• The cov() function generates the matrix, and round() is used for readability.
• Each value represents the covariance of two variables.
• Entries along the main diagonal represent the variance (σ²) of each variable with itself.

Correlation

• Covariance is scale-dependent and can be hard to interpret.
• Correlation is a standardized covariance measure describing the strength and direction of association between variables.

Population Correlation

• At the population level, correlation is denoted as ρX,Y.
• It uses the population covariance and standard deviations of X and Y.
• The correlation value always falls between -1 and 1.

Sample Correlation

• Calculated using the Pearson correlation coefficient (rxy).
• Formula involves summing the product of deviations from the mean for X and Y, divided by the square root of the product of squared deviations.
• Equivalent to dividing the covariance by the product of the measured standard deviations of X and Y.

Coefficient Calculation in R

• The cor() function in R easily computes correlation.
• Applying it to miles per gallon and horsepower yields a standardized value between -1 and 1.

Interpreting Correlation Coefficient

• Correlation ranges from -1 to 1.
• rX,Y = 1 indicates a perfect positive linear relationship.
• Recall from middle school a line can be defined as y = mx + b, where m = slope and b = y-intercept

Perfect Correlation Visualization

• When rX,Y = 1, points on a scatterplot fall perfectly on a line with a positive slope.
• Example code generates two vectors with perfect linear correlation.
• Testing with cor function confirms rX,Y = 1.

Negative Correlation

• When rX,Y = -1, there is a perfect negative linear relationship.
• Points fall on a line with a negative slope.
• Example code generates vectors for illustrating this.

Practical Considerations with Correlation

• Perfect correlations are rare in social science.

Positive Correlation Range

• When 0 < rX,Y < 1, greater values of X are associated with greater values of Y, but points do not perfectly align on a line.
• Linear regression can visually aid this relationship using geom_smooth().

Common Rules for Interpreting Correlation Coefficients

• <0.3 indicates no correlation.
• 3-.5 indicates low correlation.
• 5-.7 indicates moderate correlation.
• 7-1 indicates high correlation.

Correlation Caveats

• These are guidelines, not definitive rules.
• Interpretation is context-dependent.
• The correlation coefficient measures the presence of a linear dependency between two variables.

Correlation and Non-Linear Relationships

• The Pearson correlation coefficient measures linear dependency.
• Possible for X and Y to vary based on a non-linear function.
• Code demonstrates data following a quadratic function (y = x²).

Logarithmic Functions

• If the data is better suited to a logarithmic function, use Y = log(X) and define your relationships and data accordingly.

Spearman's Rank Correlation Coefficient Introduction

• It is a non-linear alternative to Pearson’s coefficient, it compares ranks of each variable (X & Y).
• Positive rank correlation indicates that higher values of X correspond with higher values of Y.
• Negative rank correlation indicates that higher values of X correspond with lower values of Y.

Spearman's Rank Correlation Coefficient Calculation

• Each value in X gets a ranking (R(X)) based on its size from smallest to largest.
• Smallest equals 1, 2nd smallest equals 2 etc.
• A higher ranking means the value is higher relative to the vector.

Spearman's Rank Correlation Average Datapoint

• If there are multiple datapoints with the average value, the average rank is taken.
• An example vector can be used: x= {1,2,2,3,4}
• You take the average and assign them both accordingly.

Final Spearman Calculation

• Apply pearson’s equation of the rank variables
• To assess spearman’s , follow this:
• rR9x),R(y) = ∑((R(x) -R(x))(R(y)-R(y))/√(∑(R(x)i -R(x))^2∑(R(y)i -R(y))^2

Spearman Simplification

• A perfect correlation of 1 indicates the rank-ordering of x and y are always identical.
• Use cor() function to run the spearman correlation within R.
• To confirm use method = “spearman” for clarity.

Kendall's Rank Coefficient Introduction

• An alternative approach to determining non-linear correlation.
• In this case, let n = observations for X and Y.
• Define measurements for the nth observation with pair (Xn, Yn).

Kendall's Concordance Introduction

• Count how many concordant and discordant pairs are in the data.
• Define two pairs (Xi, Yi) & (Xj, Yj), the pair is concordant if:
• Xi > Xj, & Yi > Yj.
• Xi < Xj, & Yi < Yj.

Discordant Pair

• A discordant pair is a pair in which one has a greater x value and the other has the greater y value.
• Concordant = that pair has great x and y values
• Discordant = that any one pair has great x/y values over the other

Kendall's Variables

• Let c = number of concordant pair and d= discordant pairs.
• Variable addition,
• Let x0 = number of pairs where the x values are equal
• Let y0 = Number of pairs where the y values are equal

Final Equation

• Use Greek letter “tau” which is pronounced “owwww” when you stub your toe.
• To calculate this: Th = C-D / √(c + d + x0) * (c+ d+ yo)
• Logic dictates that the numerator captures ratio of concordance to discordance

Using Spearman in R

• In R, one can use the cor() function to implement spearman.
• method= “kendall” can be used to clarify.
• Spearman should provide and tell if the data is pretty jumbled together, or if there is a set pattern.

Final Discussion : Correlation Matrix

• Analagous to covariance matrix, we can make a table that showcases all the relationships in our dataset.
• Cor() function can be used by supplying a dataframe for clarity.
• Can specify alt-correlations if the correlations are unclear (also using the cor() function)

Final Result: The Matrix

• Main direction of the matrix equals 1 (top left to top right)
• This happens because is is perfectly correlated with itself
• Correlations can also be represented with the correlation matrix.
• Corrplot library can then be generated using the correlation matrix and the cor() function.

Plot Usability

• Big blue circles show positive correlation from afar
• Big red circles represent the negative correlation
• Generally we run this correlation on continuous data, not necessarily categorical data.
• Assessing correlation is important for implementing other methods going forward!

Explore covariance: negative values indicate inverse relationships between variables. Positive covariance, like age and height in young people, shows variables move together. The sign of the term (xi - x̄) * (yi - ȳ) is crucial for determining the relationship direction.