Response and Independent Variables

• Statistical models are tools that help relate an outcome (Y) to other variables (X1, X2, ...).
• The outcome Y is also known as the response variable or dependent variable.
• The variables X are referred to as predictors, explanatory variables, or independent variables.

Model Form

• Models generally take the form:
• f() represents a function applied to predictors.
• represents the error of the model.

Model Usefulness

• All models simplify reality and never perfectly fit data.
• Two questions to consider when using statistical models:
  • How far wrong is the model?
  • How useful is the model?

Model Building

• The general step in model building include:
  • Selecting the form of the model based on data structure and patterns.
  • Fitting the model to the data using a model fitting algorithm, like a regression function.
  • Assessing how well the model fits the data, i.e., how wrong it is.
  • Using the model to answer a scientific research question.

Linear Regression Overview

• The presentation involves linear regression.
• The discussion includes:
  • When to use linear regression.
  • How to fit a linear regression to data in R.
  • How to assess model fitness in R.
  • How to extract results using R to answer scientific research questions.
• The example uses the built-in mtcars dataset in R.

Simple Linear Regression Definition

• A simple linear regression relates an outcome Y to a single predictor X.
• Linear regression attempts to fit a linear function f() such that Y = f(X).
• A simple linear function is Y = mx + b, where m is the slope and b is the intercept.
• The slope m indicates how much Y changes when X changes by one unit.

Typical Function Notation Details

• In defining a linear regression, a convention is used: This notation represents the intercept, where is the slope.
• The notation where we regress Y on multiple predictors X1, X2,...,Xn is useful.

Error Considerations

• The term "error", aka residual, recognizes the regression function may not intersect every data point in the model.
• The errors and residuals will be visualized and discussed.
• Models simplify data and don't reflect a perfect data reflection.

Linear Regression Assumptions

• Assumptions of Linear Regression:
  • Linearity: Relationship between Y and predictors X follows a linear pattern.
  • Homoscedasticity: Error/residual has the same variance (spread) at every level of X. If errors are larger for larger X values, it's heteroscedasticity.
  • Independence of Residuals: Error in one observation is not correlated with the error in another observation or a predictor's value.
  • Normality of Residuals: Unseen error in the model follows a normal distribution with a mean of 0.
  • No Multicollinearity: For multiple regression with multiple predictors (X1, X2, ...), the independent variables should not be correlated. Interpretation becomes challenging if they are.

Scatterplots Details

• Scatterplots visually inspect a linear relationship between X and Y.
• In R, the ggplot() function paired with the geom_point() function and the stat_smooth(method = "lm") function can be used to create a scatterplot,
• The stat_smooth() function creates a linear function that best fits the data
• The regression line doesn't actually touch the points.

Assumptions Assessment

• Statistical model assessment checks whether the assumption holds.
• Evaluating model assumptions is important because statistical software can run methods regardless of assumptions being met.
• It is important to assess if the method being chosen is appropriate.

Ordinary Least Squares Regression Details

• Fitting a linear regression identifies a linear function f() such that Y = f(X) fits the data well.
• This is achieved by running an optimization function which finds the best solution given some task and metric.
• The goal is to identify a linear function that's as close as possible to each data point.
• A metric is needed to measure how close the linear function is to the data.

Residuals Context

• When fitting a linear regression, a line is essentially created representing the relationship between X and Y.
• Each point represents an observed pair of X and Y values.
• Regression lines don't actually go through the points necessarily
• Residual for a given observation is how far the observed point (x,y) is from the regression line.

Residual Visualization

• Given an observation (x,y).
• Let be the value of Y on the regression line for the value of X.
• The residual is the vertical distance from the point (x,y) to the linear function.

Residuals Minimizing

• Ordinary least squares regression is fit by minimizing model residuals.
• Squaring each residual is needed because if the sum of the residuals is calculated, they add up to 0.
• For n observations:
• SSE (sum of squared error/residuals) is the regression that minimizes the value

Regression Tool

• Optimization is time intensitive.
• Computers help run many tests in a short time
• OLS regression by hand can be intensive, it's better to do this by computer
• It can easily be implemented with R

Linear Regression Fitting in R Context

• A linear regression can be fit in R using the Im() function.
• Arguments needed:
  • The regression formula takes the form outcome ~ independent variables
  • The outcome goes on the ~ left side, and the independent variables go on the right side
  • A + sign separates predictors.
  • The data.frame being drawn from
• R finds the best fitting linear function by minimizing SSE.

Linearity and Homoscedasticity Test

• A fit-residual plot can be used.
• The x-axis is the outcome variable.
• The y-axis shows the residual for each observation.
• Points should be scattered uniformly around the line.

Fit-Residual Plot Interpretation

• Ideal plot shows what linearity and homoscedasticity look like when the assumptions hold.
• Points are distributed uniformly across the horizontal line for all values of Y.

Normality Test

• QQ (quantile-quantile) plot checks normality of Residuals
• It compares observed data distribution (y-axis) to how it would be distributed if normally distributed (x-axis).
• Confidence is felt in residual normality if points follow the line.
• Divergence from the line indicates the assumption may be violated.

Independence Evaluation

• Independence is evaluated through two forms
• Residuals should be independent of the values of each predictor
• Residuala shoudl be independent of one another with no autocorreleation
• A scatterplot with residuals (y-axis) and a given predictor (x-axis) is done visually inspecting for independence (i.e. no pattern)

Autocorrelation Context

• Autocorrelation occurs when the error for a observation connects to another
• A random sample observation should be idependent, leading to independence of residuals
• Time series and nested data should be accounted for if related
• Making an assessment foor autocorrelation occurs from the study design
• Methods such as: time series and mixed effects models control any effects of autocorrelation

Multicollinearity Details

• An important assumption for multiple regression is that independent variables shouldn't be correlated.
• Use a correlation matrix to all the variables.

Variance Consideration

• VIF (Variance Inflation Factor) is the level of multi-collinearity between two models
• VIF equaling 1 indicates no multi-collinearity
• A VIF greater than 5 means that multicollinearity can be affecting performance

Variables Transforming Details

• Variables Transformation is what happens when assumptions are violated to fix predictors and responses
• Applying a math function to one variable means, we can transform it
• Variable X could can be transformed by:
  • Squaring it
  • Taking the natural log
  • Taking square root of it

Variable Transformation Applications

• When assumptions are violated, exploring transformations is important.
• Normalize the data
• Try to use intuition on what has potential
• Look at the distributions of values, this helps

Log-Transforms Details

• In some instances we can use Log-Transforms
• Common to observe sample data skewed right
• Set a concentration above the mean and below it
• Log transform can help reel in skew and help the model to work better

Model Interpretation

• Transforming a variable will change the model
• De-transforming to contextualize is good
• Interpreting results of a linear regression helps reflect transform choice

Outliers Considerations

• A linear regression isn't robust with outliers
• An outlying point can impact model performance in a big way.
• Extreme outliers should potentially be removed
• Trying to identify is important

Cook's Distance

• Cook’s distance shows how much the fitted values of Y are changed by an observation.
• For each observation, Cook’s distance is calculated.
• A rule of thumb: Observations with a Cook’s distance 4 times greater than the mean Cook’s distance.

Checking Assumptions

• When assuming linear regressions
• The model tries to answer how well it fits for our research
• Two apsects to this:
  • Evalute the relation between X1/X2 and Y, or the response
  • Diagnoise well the data fits, this can increase relative data

Result Details

• The summary() function in R shows a lot about how the model works
• Regression in R happens once we examine infomation
• A table will display, using this function
• Pieces of inofmation answer our research question

Intercept and Slopes Context

• When trying to fit linear regression with: the formula
• There is code to support the equation from R

Hypothesis Evaluation

• Each variable defines the linearr elationship
• Increases in certain aspects, can change the effects
• Random chance or population affects this?

Null Hypothesis Test

• A null hypothesis is assumed in inferential statistics
• A beta coefficient to X will euqal zero
• Probability gets an observed slope with this

Regression Slope

• Captures the standard error of the regression on the slope for testing
• R captures this automatically , using the dispersion of the restiduals

Standard Details

• Captures how well the data is with regression
• Closer the value is, the smaller the falls are
• N-2 degrees here will calculate the dispersion
• Remaining can inform this dispersion
• Data points affect the standard error, if we fit a linr through them

Running A T-test Details

• To run this we assume certain aspects
• A null hypothesis is also assumed
• Mean and residuals are distributed normally over this
• Running the T-Test requires a certain form to it
• A T Value formula creates a sample coefficient

T-Table Details

• Here we use the distribution test
• Comparing a value to a distribution with n-2 degrees
• Our example showcases with one degree
• Right portion uses Two-Tailed Value

Significance Considerations

• If it this is useful this provides results
• Indicating between X and Y is useful
• The findings provides if the hypothesis failed

Point Estimate

• Estimate helps us get a one united increase in something
• Random data here supports
• Parameter works well with random samples

Confidence Intervals

• We do computations for all the values
• This has many details tied to the points or facts

Research Paper Considerations

• Most documents prefer:
  • Point of Estimate
  • Error of Std
  • Interval
  • Rounded P Value
• Automate the creation
• Model data to make assumptions

Determination Coefficient

• The data model explains the test here
• The predicted Y and value of Y represents it
• Values closer to 1 indicate the model will explain the most
• Square correlation equals that

Adjusted Data

• Adding data means we expand with it
• Adjusted and K test equal the predictions and degrees of freedom
• Model predictions are good to see

ANOVA

• Can the model provide the details for most results?
• Data points needs the info.
• This works well in hypothesis testing

Data Division

• ANOVA logic
• The test is helpful within tests
• That is how test for slopes exists
• Other footnotes report regressions
• Tests make most test null

Explains response and independent variables in statistical modeling. It includes the form of a statistical model. It also explains the significance of model building.