Understanding Correlation and Correlation Coefficients

Introduction to Correlation and Correlation Coefficients

• Objective is to describe how the Pearson experiment correlation coefficient identifies possible correlations between variables
• Correlation does not equal causation
• Finding correlations between variables can be useful for analysis
• Two specific coefficients introduced: Pearson and Spearman

Graphical Representation of Correlation

• Joint Distribution Graph plots two variables against each other
• In an example, as X increases, Y increases which indicates a possible positive correlation
• Using another example, there is no clear correlation, as values of X do not predict values of Y
• In a further example, there is a reverse correlation, where as X increases, Y decreases
• A correlation coefficient is needed for statistical backing of the relationship
• Pearson correlation coefficient is a statistical measure to quantify the relationship between the two variables
• The coefficient helps in identifying the strength and direction of a relationship

Types of Correlation Coefficients

• Pearson (r) measures the linear relationship between two continuous variables
  • Continuous variables are numerical and can take any value within a specified range
    • Both X and Y must be continuous
  • Uses a parametric test
    • Involves assumptions about the parameters of the population
    • Associated with a parametric distribution, typically a normal distribution
  • Ranges from -1 to +1
    • A coefficient of -1 represents a perfect negative (downward) correlation
    • A coefficient of +1 represents a perfect positive (upward) correlation
    • A coefficient of 0 represents no correlation
  • Measures how data points fit a linear line, and how confidently the next point can be predicted
    • The coefficient tells how closely the data points fit, not the slope
    • A perfect correlation of 1 or -1 means the data points fall perfectly along the line
    • A good fit means that a line drawn through the data points has a high Pearson coefficient, for example 0.8
• Spearman (ρ or "rho") measures the relationship between two ordinal variables, or when the data is not continuous
  • Ordinal data is not concerned with the exact difference between values, but rather the order (ranking)
  • Used to measure if the values follow a sequential relationship
  • Ranges from -1 to 1, similar to Pearson
  • If each value in X corresponds to a higher value in Y in a sequential manner, Spearman correlation would be 1, indicating perfect rank correlation
• Kendall rank correlation coefficient
  • Assesses the strength and direction of a relationship between two variables
  • Useful when data may not meet the assumptions of other correlation methods, particularly for sample sizes or when dealing with ordinal data
  • Is less sensitive to ties (when multiple data points have the same value)
  • Gives a more stable and consistent measure of correlation when the data is sparse or when there are ties
  • It performs better with small datasets because it is more stable
  • A non-parametric test, meaning it doesn't assume that the data follows any particular distribution
  • Based on the ranks of the data, rather than their actual values
  • Assesses the relationship between two variables by comparing the ranks of data points, rather than their raw values
  • Scale ranges from -1 to +1
    • τ = +1 indicates a perfect positive relationship
    • τ = -1 indicates a perfect negative relationship
    • τ = 0 indicates no relationship between the variable

Understanding the Difference Between Correlation and Causation

• Correlation does not imply causation, it indicates that two variables are related but it does not prove one causes the other
• A correlation between ice cream sales and shark attacks doesn’t imply that buying ice cream causes shark attacks
• Both could be influenced by warmer weather

Key Points

• Pearson is used for continuous variables with a linear relationship
• Spearman is used for ordinal or non-continuous data, focusing on rank-based relationships (non-parametric)
• The Pearson coefficient quantifies the goodness of fit for linear relationships
• The Spearman coefficient looks at sequential relationships and rankings rather than exact values.
• Focus is on how confidently we can predict future values based on existing data points
• Causality is not assumed, correlation only points to a relationship between variables