Chi-squared Test for Categorical Data

The Chi-squared test assesses if observed differences in categorical data are due to random chance or a meaningful pattern.
Karl Pearson developed the Chi-squared test, introducing it in 1900 as a foundational method in modern stats.

Expectations vs. Observations

The Chi-squared test maps observed patterns in categorical data onto a theoretical distribution for significance testing.
The Chi-squared Test of Independence determines if two categorical variables, X and Y, are related (dependent)
It cannot map categorical data values onto some distribution effectively without this test

Income Level and Smoking Status Example

A study recruits 400 people to examine the relationship between income and smoking status.
Income is measured categorically (<$20k, $20-50k, >$50k) and so is smoking status (current/not current smoker).
The sample reports: 100 people earn <$20k, 200 earn $20-50k, 100 earn >$50k; 100 are current smokers, and 300 are not.
A cross-tabulation is used to build the framework for analysis.
The total row/column in the cross-tabulation shows the total people in each category, with the bottom right displaying the total sample size.
Cells within the table represent the number of people reporting each combination of income and smoking status like, annual income below $20k and reports not current smoker.
Assuming independence (null hypothesis), values of both variables are evenly distributed across groups.
The expected value in each cell can be determined by using the equation: (nrow * ncolumn) / n
nrow is the total participants in a given row
ncolumn is the total in a given column
n is the total number of participants.
Example: nrow = 100, ncolumn = 300, and n = 400.
The number of people reporting income less than $20k who don't currently smoke is (300*100)/400 = 75.
Distribution of income is assumed to be the same among smokers and non-smokers.
Distribution of current smoking is expected the same across each income level (75% not, 25% current).
The expected distribution of two variables are created assuming independence.
Then, the actual observed values get recorded.
A greater observation of people earning less than $20k currently smoke than expected, plus a smaller proportion earning more than $50k.
The difference Erow,column and Orow,column represents the signal and shows how observed data differs when independent.
Summing the differences between expected and observed values in each cell equals 0, making it not a useful metric.
Squaring any value makes the result positive.
With the sum of several squares, the result is only 0 if the data perfectly matches the expected values.

Equations

The equation for the new sum is: Σ(Orow,column – Erow,column)^2
The signal strength is still dependent on study sample size where more people sampled with an identical proportional distribution of observed results.
Standardize the signal calculation by dividing the difference between each observed and expected value by the expected value.
Now, each term represents the difference between observed and expected values relative to the expected value.

Equation

This can be mathematically represented as: X^2 = Σ ((Orow,column - Erow,column)^2) / Erow,column
Standardized value represents signal strength with no signal existing.
x^2 has (rows – 1) * (columns – 1) degrees of freedom, very important because it allows use metric to asses probability of our observed data when not related.
We can use this to determine if the two variables are not related

Chi Square Distribution

Normal Distribution arises from understanding certain random natural phenomenon.
A normally distributed variable is where the mean is the value to observe, where values closer to it are more likely with values less than mean equally likely to occur.
Standard normal distribution (Z-distribution) is used and represented.
Squaring the values of the Z-distribution yields the resultant distribution.
Squaring the distribution means any point (x,y) in the Z-distribution gets projected (x^2, y^2).
Distribution is the chi-squared distribution with 1 degree of freedom, described by this equation. X^2
By definition ~68% of observations of a variable assumed the Z-distribution will fall between -1 and 1.
Squaring a value between –1 and 1 results in a value between 0 and 1.
With the X^2 distribution the Z distribution we can understand that ~68% observations will fall between 0 and 1.
If we have a variable X following the standard normal Z-distribution, 𝑋^2 follows the 𝑥^2 distribution.

Equation:

𝑌= ∑𝑋_𝑖^2 with the i running from 1 to k. is then understood to be distributed according to Chi^2 with k degrees of freedom.
Degrees of freedom are increased, the curve shifts drastically where values closer to 0 are likely for one standard.
Chi^2 describes the probability distribution of a varable represented in the sum of K independent squares.

Chi Squared Test of Independence

Equation:

𝑥^2= ∑ ((𝑂_𝑟𝑜𝑤,𝑐𝑜𝑙𝑢𝑚𝑛 −𝐸_𝑟𝑜𝑤,𝑐𝑜𝑙𝑢𝑚𝑛 )^2)/𝐸_𝑟𝑜𝑤,𝑐𝑜𝑙𝑢𝑚𝑛

A sum of squares allows to identify a sum of squares because we have a sum of squared values.
Chi squared distribution the sum of K normally distributed variables used in the equation.
If variables independent, most likely value is the observed count equals the expected.
That these differences follow a normal distribution with each row column combination, summing together nrow * ncolumn.
BUT, Chi squared distrubtion is in relation to K independent normally distributed variables.
It cant be fully independent because more people in cell of our table will influenced by many people remain to be distributed.
Have to start inserting observed number and report not currently smoking.
We finally compare our test statistic to the Chi-squared distribution with 2 degrees of freedom.
Need to know what probability of observing a value of 48 or more extreme, assuming the null hypothesis is true (independent).
𝑋^2
=∑ ^( (𝑂_ 𝑟𝑜𝑤,𝑐𝑜𝑙𝑢𝑚𝑛 −𝐸_ 𝑟𝑜𝑤,𝑐𝑜𝑙𝑢𝑚𝑛 )^2)/𝐸_𝑟𝑜𝑤,𝑐𝑜𝑙𝑢𝑚𝑛 (aka the 𝜒^2 distribution with 2 degrees of freedom). - We can easily calculate the appropriate number of degrees of freedom by calculating (𝑟𝑜𝑤𝑠 − 1) ∗ (𝑐𝑜𝑙𝑢𝑚𝑛𝑠 − 1) , where rows is the number of rows in our table and columns is the number of columns in our table. - In this example we can see this would equal (3 − 1) ∗ (2 − 1) = 2 ∗ 1 = 2. -
=(50 − 75)^2 /75+(50 − 25)^2/25+ (160 − 150)^2/150+ (40 − 50)^2/50+(90 − 75)^2/75+ (10 − 25)^2/25
Now, the final step of our test is to compare our test statistic ( χ^2 = 48) to the χ^2 -distribution with 2 degrees of freedom, or χ_ 2^2 . Specifically, we want to know what is the probability of observing a value of 48
or a more extreme value, assuming the null hypothesis is true (that our two variables are independent).
P(χ^2 ≥ 48|χ_ 2^2 )
we use this for the calculation by calculating the area under the curve.
The Chi^2 Test of Independence like so with 2 categorical variables, assessed to be the relations.

Hypothesis

H0: X and Y are independent of one another.
HA: X and Y are dependent on one another.
Create a table representing the cross-tabulation of X and Y by representing the value in the level of the test.
The equation is 𝐸_𝑥,𝑦=(𝑛_𝑋 ∗𝑛_𝑦)/𝑛
Where nx represents the total number of participants reporting level x from X and represents the value for from Y.
After the comparison, We will be able to look at the test statistic in the test.
Compare a distribution appropriate variable that can be in levels.

Assumptions of the 𝑥^2 Test of Independence

X and Y are both categorical.
The levels of each variable X and Y are mutually exclusive.
Each observation is independent - in other words, our data comes from a random sample of independent observations. 4.The value of Ex should be 5 or greater in at least 80% of table cells and Ex must be at least 1 for every cell.

Running the test in R allows for the matrix function creates what like the data.

Goodness of Fit Test

Assessing if distribution of single categorical variable mathches some pre defined dstribution H0 : The distribution of X fits the predetermined distribution HA : . The distribution of X does not fit the distribution. X

Value 𝐸𝑥= (𝑛)( population)