Modules 10-21: Correlation and Bootstrap

Module 10: Correlation and Bootstrap

• cor(x,y) calculates the correlation between 0 and 1; only use when the relationship is linear.
• Correlation is the strength of the linear relationship between two quantitative variables.
• Jitter plots smear categorical data, on a quantitative scale, to visualize the density of observations at each intersection of values, using plot(jitter(x, factor=1), jitter(y)).
• table(x,y) calculates the frequency of each pair of responses.
• The frequency of a table can be turned into a size-proportional scatterplot by assigning freq <- c(table(x,y)), then plotting with plot(x, y, cex = sqrt(freq)), and superimposing frequencies text(x, y, freq).
• Bootstrapping correlation and regression slopes involves calculating the correlation cor1 <- cor(x,y), testing if the correlation is nonzero cor.test(x,y), fitting a linear regression model lm1 <- lm(x~y), and extracting coefficients of the line lm1$coef which returns the intercept and slope.
• Bootstrapping is done by sampling rows with replacement from the original dataset, ensuring the same number of observations, then calculating the correlation and slope for the "fake" dataset to determine confidence intervals.

Module 11: Correlation, Permutations, Multiple

• The Permutation Test assesses correlation by checking the p-value cor.test(cry$Crying, cry$IQ).
• Outliers can artificially inflate correlations; so a permutation test is performed where one variable is permuted and then recalculated.
• To use the permutation test first permute one variable fakeIQ = sample(IQ) then bind the natural order cbind(crying, IQ) and the random order cbind(crying, fakeIQ).
• The p-value, representing the likelihood of observing the true correlation if there is no association, is then calculated.
• To make CorResults for the permutation test assign a number of samples, for example, 10000, by setting corResults <- rep(NA, n-samp).
• To test for correlation run a loop where: for (i in n-samp) { corResults [i] <-cor(crying, sample (IQ)) }.
• Calculate a 2-sided p-value for fake correlations by finding the tme cor <- mean(abs(corResults) >= abs(cor(crying, IQ))).
• WB2 <- WB2[complete.cases(WB2), ] changes the dataset to include complete entries for variables
• cor(WB2[,-1]) outputs correlations between all columns, made easier to visualize in a plot.
• corrplot(cor(WB2[,-1]), method = "ellipse") generates a plot displaying the direction and strength of correlations.
• Matrix plots can be used to to check for linearity with multiple scatterplots plot(WB2[,-1]).
• Taking the log may help linearize data.
• Other methods include chart.Correlation(WB2[,-1]), corrplot.mixed(cor(WB2[,-1]), ...other modifiers...).
• Pearson Product-Moment Correlation is calculated using cor.test(ctTEMP$Min, ctTEMP$Year).

Module 12: Regression

• The theoretical regression model is My = Bo + B₁X.
• The actual values are represented as Y₁ = Bo + B₁X₁ + Eᵢ, where there are independent observations from a normal distribution with a mean of 0 and some common standard deviation.
• Before performing regressions, verify the assumptions of linearity and normal distribution of errors.
• To account for missing values use code like crosby <- na.omit(crosby).
• To fit the model you can use lm1 <- lm(Mintemp ~ Year).
• To show coefficients of line you can use lm1$coefficients.
• You can find the confidence interval for the true slope using confint(lm1, 'Year').
• You can also use bootstrapping here, sample with replacement, calculates a new correlation, saves scores into a vector, and get a CI of those scores.
• The boot CI will be wider than the theoretical CI.
• Quadratic Models can be create using the code lm2 <- lm(y ~ x + x^2) and adding a curve to the plot using the code points(x, lm2$fitted.values, type = "l").
• For bootstrapping quadratics begin with finding the number of rows using N <- nrow(data).
• You can use the code bResults <- matrix(rep(NA, 3*n-samp), ncol = 3) to create a matrix.
• You can use the code S <- sample(1:N, N, replace = T) to get a sample. Assign the fake year fakeYear <- Year[s] and the fake year squared fakeYearSq <- fakeYear^2, and the fake temp fakeTemp <- Mintemp[s].
• The code lmtemp <- lm(fakeTemp ~ fakeYear + fakeYearSq) can get the results bResults[i,] <- lmtemp$coef.
• Finally get the get the CI using the code ci-line <- quantile(bResults[,2], c(.025,.975)) and ci-quad <- quantile(bResults[,3], c(.025,.975)).
• Linear regression aims to find the best-fitting line (or hyperplane) that models the relationship between a dependent variable 𝑌 and one or more independent variables 𝑋
• The observed values of 𝑌 are equal to the mean function plus a random error: 𝑌𝑖=𝛽0+𝛽1𝑋𝑖+𝜀𝑖
• The errors 𝜀𝑖 are assumed to be independent and identically distributed random variables from a normal distribution with mean zero and constant standard deviation: 𝜀𝑖∼𝑁(0,𝜎)
• The aim is to estimate the following unknown parameters: 𝛽0,𝛽1,𝜎, where 𝛽0 is the true intercept, 𝛽1 is the true slope, and 𝜎 is the true standard deviation of the errors.

Module 13: Regression 2

• For a 95% confidence interval for average minimum temp in 2100, a prediction band is needed, which is wider than confidence intervals since it accounts for the range of potential individual data points.
• For a 95% confidence interval for the regression line in year 2100, use a confidence band.
• 95% confidence means the regression line is between blue lines.
• 95% confidence means all future data will fall between the red lines.
• R-squared measures the proportion of variance in y explained by the regression model.
• Influential points and outliers (large residuals) can affect the regression line.
• Cook's distance quantifies the influence of a data point; a larger value indicates greater influence.
• Standardized residuals are calculated as the residual divided by the sample SD.
• Heteroskedasticity exists if the standard deviation of residuals is non-constant.
• Use a transformation to deal with heteroskedasticity by making ln() or sqrt() transformations; look for a fanning pattern in the residual spread.
• Add a regression line to a plot using abline(lm1$coef, lwd = 3).
• New temperatures can be predicted until 2100 using the code newyear <- seq(min(Year), 2100, by=1) and assigning the data frame nd <- data.frame(Year = newYear).
• Use the code confbands2 <- predict(lm1, interval = "confidence", newdata=nd) and predbands1 <- predict(lm1, interval = "prediction", newdata=nd) to get results.
• The code plot(mod1, which=4) generates a plot of Cook's distance.
• The code text(x,y, vector of text to show, pos = 1) superimposes text on points.
• Create a normal quantile plot using the function qqPlot(rstudent(lm1)) which generates a norm-a plot of studentized residuals.
• Good visualization will result in no pattern.
• Using the library olsrr can give useful residual plots.

Module 16: Multiple Regression and Continuous Predictors

• In multiple regression, the theoretical model equation: Y = Bo + B₁X₁ + B₂X₂ +...+BₙXₙ.
• Bₙ represents the change in Y for every unit increase in Xₙ; there are three assumptions: linearity, constant variance, and the predictors are not strongly correlated.
• Predicted values are calculated ŷᵢ = b₀ + b₁X₁ + b₂X₂ + ... + bₙXₙ, and residuals are calculated eᵢ = Yᵢ - ŷᵢ.
• Multiple R² indicates the amount of variability in Y explained by multiple regression.
• Forward stepwise regression aims to create a simpler model by starting with the best predictor and iteratively adding significant predictors.
• R² value always increases when more are added, which is a problem.
• Adjusted R² adds a penalty for the number of terms; always less than R² use Adjusted R² .
• Akaike Information Criterion is useful for smaller models and is only used when comparing different models.
• Cp statistic helps identify the simplest model.

Module 18: Multiple Regression and Categorical Predictors

• Assesses whether things are getting warmer over time by checking if in regression slope > 0
• Variables may be categorical and non-orderable.
• R automatically creates reference variables, omitting one level of each categorical variable from the model.
• Assumptions of heteroskedasticity are tested; you can do Box-Cox transformation if assumptions are not met.
• levels(var) displays the levels of a variable.
• Multiple boxplots can be displayed boxplot(TMIN ~ Month, data = ctTEMP).
• Code to Change order of levels includes steps such as levels (ctTEMP$Month) <- c("01-Jan", "02-Feb",..., "12-Dec") and sort(levels(ctTEMP$Month).
• To create a linear regression model w/ multiple categorical predictors use the code lm1 <- lm(TMIN ~ Year + Month + Name, data = ctTEMP).
• Code to Try different reference levels is ctTEMP <- within(ctTEMP, Name <- relevel(Name, ref = "STORRS")).
• Test normality of residuals and heteroskedasticity using the hist(lm1$residuals) function.
• qqPlot(lm1$residuals) plots the values of the residuals.
• plot(fitted(lm1), rstudent(lm1)) plots the fit of the residuals.

Module 19: ANOVA

• ANOVA serves as a regression model with at least three levels of a categorical variable. A t-test can be used if there are only two.
• The ANOVA assumptions are that the populations are from normal distributions, constant standard deviation/variance between groups, and all observations are independent.
• ANOVA assumes a overall mean is modified by a "treatment effect".
• It Quantifies the amount of variation observed in the dataset due to group identification.
• To use the code CODE fit ANOVA model use code such as aov1 <- aov(cooperation ~ emotion).
• Conducted is pairwise.t.test(cooperation, emotion, p.adjust = "none").
• Turkey's Honestly Significant Difference Test is used using the code TukeyHSD(aov1).
• ANOVA has a total sum of squares using the equation TOTAL sum squares = SSgroups + SSerrors.
• ANOVA also has a null hypothesis where the mean in each groups are around the same.
• ANOVA also has an alternative hypothesis means Do vary across groups.
• ANOVA uses the code F=mean square group/ mean square be eror.

Module 20: Other Tests

• Tests for equality of variances include Ho: σ₁=σ₂=...=σₙ and HA: one σᵢ different.
• Assumes normality of group distributions and a p-value says if one SD is different.
• Uses bartlett.text.
• Levene's Test does NOT assume normality of variances but needs more robust data; test uses levene.Test.
• Welch's ANOVA is for normal populations with unequal variances, and uses oneway.test(y~x).
• Kruskal-Wallis Test assumes that there are NO DISTRIBUTIONAL ASSUMPTIONS and the HO states group medians are same while HA states at one group the median is different, test kruskal. test
• Two-Way ANOVA is made of 2 categorical predictors + their interaction term with the same assumption of equal variances; uses, aov(y~ x + z + x*z)..
• Uses interaction. plot(x, groups, response.
• observation = overall mean + row factor effect & column factor effect+ interaction effect & residuals / errors.

Module 21: ANCOVA, GLMs

• A generalized Linear Model is any kind of linear model ex regression, one-way Anova, tow-way Anova, 1 of 2 sample I tests.
• Model can be expressed in the form Y = Bo + BiX. + B2X2 + ... + βnxn + €.
• To calculate Type I Sum of Squares (Sequential) you calculate the Amant of model explained by a factor, by is affected by terms BEFORE IT.
• You can find use the code to use other types of Sum of Squares such as to create ANOVA(..., type = "III").

Additional Concepts

• To Prepare for exams review class material modules 16 to 22, review labs from 5 onwards, review labs and quizzes and complete a sample exam.
• Exams will test: writing basic R code, knowing and understanding R functions, reading R code, interpreting R output, and applying substantial knowledge of key concepts.
• Interpret continuous predictors (e.g., V241324, V241325, V241330x) such that a 1-point increase in the independent variable corresponds to a 𝛽 point change in the dependent variable (e.g., V241157, Trump feeling thermometer), holding all other variables constant.
• For categorical predictors (e.g., V241227x, Party ID), coefficients represent the difference in outcome between each category and the baseline (reference) category, usually Independent.
• The intercept represents the predicted value of the dependent variable when all continuous predictors are equal to 0, and all categorical predictors are at their reference (baseline) category.
• Consider whether the intercept is meaningful in the context of the variables and if a value of 0 on a 1–7 scale has a real-world interpretation.