Understanding Covariance in Health Research

• CHS 729 Week 6 covers covariance and correlation, path diagrams/conceptual models, and the anatomy of a methods section.

How 2 Variables "Vary Together"

• Wanting to know the relationship/association between two variables is common.
• Continuous variables X and Y are understood by the method in which they "vary together".
• Changes in variable X are associated with changes in variable Y, and vice versa.
• Example: As a person gets older (X=age), they develop more gray hairs (Y=amount of gray hair); thus, these variables "vary together".

Covariance

• Covariance is related to how variables X and Y vary together.
• Positive covariance means larger values of X are associated with larger values of Y.
• Negative covariance means larger values of X are associated with smaller values of Y.
• Understanding how variables "vary together" is fundamental in statistics and is useful when considering
  • How exposure to an intervention is associated with changes in an outcome
  • How exposure to a risk/protective factor is associated with an outcome
  • How risk and protective factors are associated with one another

Measuring Covariance

• For each participant i in a sample, the distance of their values from the mean values of X and Y is calculated, and then the average is taken.
• This term contains two pieces of information:
  • How far participant i value of X is from the sample mean
  • If the participant i value of X is greater than or less than the sample mean
• If a participant i value of X and Y are both greater or less than the sample average, they contribute a positive value to the total covariance.
• If a participant i value of X and Y are on opposite sides of their respective means, they contribute a negative value to the total covariance.
• Positive covariance indicates that values of X greater/less than the sample mean are associated with values of Y greater/less than the sample mean.
• Negative covariance indicates that values of X greater/less than the sample mean are associated with values of Y less than/greater than the sample mean.
• Covariance only reflects that variables X and Y may vary together, but it does not explain why.
• Covariance is not standardized, so changing variable units can change the magnitude of the measured covariance.
• Covariance between vehicle weight in pounds and miles per gallon has a different value than vehicle weight shifted to grams.

Measuring Covariance in R

• Covariance can be measured in R using the cov() function.
• The function takes two arguments, with each vector representing covariance between two variables.
• This function returns the value of covariance.
• Covariance values can range from negative infinity to positive infinity.
• There is negative covariance between fuel efficiency of vehicles (mpg) and horsepower (hp).

Visualizing Covariance

• Ggplot2 library is useful for visualizations.
• In a visualization of negative covariance, points follow a pattern from top-left to bottom right.
• Ggplot has many built-in themes.
• These built in themes allow for altering the appearance of plots.
• These can also be customized upon becoming familiar with the package.

Covariance Matrix

• Calculating the covariance between all dataset variables can be useful and create a covariance matrix.
• Each matrix value represents covariance between row and column variables.
• The main diagonal shows the variance of each variable.
• Supply the data to the cov() function to create a covariance matrix.

Correlation

• Covariance is useful but can be difficult to interpret since it is not standardized.
• Correlation is a standardized version of covariance for continuous, normally distributed variables.
• Correlation will always result in a value between -1 and 1.

Pearson's Correlation Coefficient

• Calculating correlation between normally distributed variables X and Y in sample data can be done by computing Pearson's Correlation.
• This is done in R by using the cor() function.
• Pearson's Coefficient measures the linearity of the relationship between variables.
• A correlation of 1 means a function y = mx + b perfectly describes the relationship of X and Y.

Interpreting Correlation Scores

• A correlation score’s strength is interpreted subjectively.
• It depends on context, applied methods, and variable nature.
• Heuristic rules:
  • <0.3 indicates no correlation
  • 0.3-0.5 indicates low correlation
  • 0.5-0.7 indicates moderate correlation
  • 0.7-1.0 indicates high correlation
• The Pearson's correlation assumes that:
  • The relationship between X and Y is linear in nature
  • There are no severe outliers
  • X and Y are normally distributed

Spearman's Rank Correlation

• One option when variable relationship is not linear is to use Spearman's Rank Correlation.
• This assesses the rank-ordering of data by assessing if the largest values of X are associated with the largest/smallest values of Y.
• X and Y are transformed into rankings of R(X) and R(Y) to calculate Spearman's Rank Correlation.
• With R(X), the smallest value of X is assigned a value of 1, with the second smallest assigned a value of 2, and so on.
• Once variables X and Y are converted into R(X) and R(Y), the Pearson coefficient ranks are taken.
• A perfect value of 1 shows X and Y have perfect rank-ordering, meaning the smallest value in the ranks corresponds to the smallest data values.
• A value of -1 indicates that value of X corresponds to the largest value of Y, etc.

Kendall's Rank Coefficient

• Alternative coefficient is Kendall's rank.
• Define the measurement with the pair, letting (Xi, Yi) be measurement with pair i.
• Identify the number of concordant and discordant pairs.
  • A pair is concordant if Xi{less than}Xj and Yi{less than}Yj or if Xi>Xj and Yi>Yj
  • Let C equal the total number of concordant pairs
  • A pair is discordant if Xi{less than}Xj and Yi>Yj or if Xi>Xj and Yi{less than}Yj
  • Let D equal the total number of discordant pairs
• Identify pairs that have an equal X value (but a different Y value) and pairs that have an equal Y value (but a different X value)
• Calculate the Kendall's coefficient.

Correlation Matrix

• Compared to a covariance matrix, a correlation matrix is useful for assessing data associations and can be made using R from a data frame
• Easily specify alt-correlations.

Correlogram

• Creating a visual correlation matrix can help identify patterns of correlation.
• Used in some methods to ensure variables are not correlated
• It can be a useful starting place when there is the need to assess correlation between variables.

Conceptual Models

• It is useful to have a set of tools to visually represent our research questions.
• Conceptual models are important for assessing associations, causal pathways, moderation, and mediation.

Squares and Arrows

• Measured variables are represented with squares.
• Arrows indicate potential relationships explored between variables.
  • A bi-directional arrow indicates an association.
  • A one directional arrow indicates a causal pathway, although it does not mean true causation can be assessed

Arrows, Justification, and Questions

• Arrows implicitly capture a research question.
• Conceptual model should be constructed so as to capture many study questions in a simple visual.
• It can be easy for others to argue that education level directly impacts income, rather than a simple association.
• The structure of a conceptual model should be justified.
• Conceptual models may demonstrate a mediation model.
• A mediation model increases clarity.
• It also defines the questions to be answered.

Moderation

• Moderation can be depicted by drawing an arrow towards another arrow.
• Moderation occurs when the relationship between education and smoking status is dependent on the third variable value, such as income, that has a moderate effect on the relationship.
• The impact of education level on smoking may be more extreme when income level is low, but when it is higher, the impact of education on smoking may become less important.
• Conceptual models can be simple and tell stories that can take paragraphs to write.
• These can also be really complicated, clarifying the understanding of a phenomenon.
• Path diagrams are foundational for structural equation modeling and capture the impact of error and measure latent variables.
• Becoming familiar with conceptual models and path diagrams is an important step to adopt SEM techniques in the future.
• A latent variable cannot be observed, but we can capture it via observed variables as shown in a path diagram using factor analysis.

The Anatomy of a Methods Section

• Methods sections tell the reader what was done.
• In experimental science, this enables other scientists to replicate findings, which calls an experiment’s validity into question if one cannot do so.
• Conceptual replication helps others understand the work that was done.
• The steps of a Methods section include:
  • Study Sample: Where data came from, when and how it was collected, who funded the study, who was the parent study's target population and what was its purpose.
  • Measures: Should describe variable use and the tools used to collect them, how data was coded, and alteration steps that were taken.
  • Analysis: The statistical analysis run, including methods used, why they were used, and what results were extracted from them.
• Given the raw data, it should be like a recipe to re-run the analysis.
• Methods should convince someone that the approaches used are both appropriate and rigorous.
• Ensure that the methods section frames what the results section will cover.

Covariance measures how two variables change together. Positive covariance suggests that as one variable increases, the other tends to increase as well. This concept is useful in research settings.