Regression Analysis Basics

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

What is the primary purpose of standard multiple regression?

To identify the most significant independent variable

To estimate the correlation coefficient between two variables

To predict a dependent variable using two or more independent variables (correct)

To control for the effect of extraneous variables

What is the term for a variable that is not of primary interest in the analysis but is included in the model to control for its effect?

Independent variable

Explanatory variable

Control variable (correct)

Dependent variable

What is the difference between the observed value and the true value in a regression analysis?

Shared variance

Residual

Error (correct)

Unique variance

What is the term for the variability in a dependent variable that is explained by multiple independent variables simultaneously?

Shared variance Signup and view all the answers

What is the purpose of the best-fitting line in a regression analysis?

To identify the relationship between the independent and dependent variables Signup and view all the answers

What is the range of the regression coefficient R2?

0 to 1 Signup and view all the answers

What is the term for the change in the dependent variable for one unit change in an independent variable, holding other independent variables constant?

Standardised slope Signup and view all the answers

What is the general linear model equation?

data = model + error Signup and view all the answers

What is the purpose of reporting R2 change in hierarchical regression?

To explain the added variance in the DV at each step Signup and view all the answers

What is the assumption of homoscedasticity in regression analysis?

The variance of the residual is the same for any value of the IV Signup and view all the answers

What is the purpose of the partial correlation coefficient in regression analysis?

To measure the correlation between the predictor and outcome variable while removing the shared variance with other predictors Signup and view all the answers

What is the goal of selecting the 'best' model in regression analysis?

To minimize the residual mean square Signup and view all the answers

What is the purpose of the semi-partial correlation (Part)sr2 in regression analysis?

To measure the correlation between the predictor and outcome variable while removing the shared variance with other predictors Signup and view all the answers

What is the purpose of the outlier score in regression analysis?

To identify unusual scores on the IV Signup and view all the answers

What is the purpose of the mediation analysis in regression?

To identify the indirect pathway association between the IV and DV Signup and view all the answers

What is the purpose of the standardized coefficient in regression analysis?

To express the slopes of the regression line in standard deviation units Signup and view all the answers

What is the assumption of linearity in regression analysis?

The relationship between the IV and the mean of the DV is linear Signup and view all the answers

What is the purpose of the hierarchical regression model?

To test the importance of different constructs Signup and view all the answers

What is the purpose of using bootstrapping in mediation analysis?

To test the significance of the mediated pathway Signup and view all the answers

What is the definition of a moderator variable in moderating regression analysis?

A variable that influences the relationship between the IV and DV Signup and view all the answers

What is the purpose of using a Sobel test in mediation analysis?

No longer used, as it requires high N and is too conservative Signup and view all the answers

What is the difference between an additive and interactive model in moderating regression analysis?

Additive models have independent effects, while interactive models have conditional effects Signup and view all the answers

What is the purpose of using variable centring in moderating regression analysis?

To create a mean of 0 for the continuous variable Signup and view all the answers

What is the minimum sample size recommended for conducting moderated regression analysis?

150 participants Signup and view all the answers

What is the purpose of using the Johnson-Neyman test in moderating regression analysis?

To examine the effect of multiple levels of a continuous moderator variable Signup and view all the answers

What is the definition of a covariate in moderating regression analysis?

A variable that is used to control for extraneous variables Signup and view all the answers

What is the purpose of using the pick-a-point technique in moderating regression analysis?

To create a figure to describe the association between the variables Signup and view all the answers

What is the assumption of homogeneity of regression in moderating regression analysis?

The covariate has the same effect at each level of the moderator variable Signup and view all the answers

What is the purpose of the omnibus test in ANOVA?

To test for an overall experimental effect Signup and view all the answers

In a repeated measures ANOVA, what is the purpose of controlling for individual error?

To separate the error term for each participant Signup and view all the answers

What is the difference between a main effect and an interaction effect?

A main effect is the influence of one IV on the DV, while an interaction effect is the joint effect of multiple IVs Signup and view all the answers

What is the purpose of the Huynh-Feldt correction in ANOVA?

To correct for sphericity violations in repeated measures ANOVA Signup and view all the answers

What is the purpose of the Levene's test in ANOVA?

To test for homogeneity of variance between groups Signup and view all the answers

What is the purpose of the R-squared change in ANOVA?

To determine the contribution of each independent variable to the explanation of the dependent variable Signup and view all the answers

What is the difference between an ordinal interaction and a disordinal interaction?

A disordinal interaction has a crossover effect, while an ordinal interaction does not Signup and view all the answers

What is the purpose of the Mauchly's test in ANOVA?

To test for sphericity in repeated measures ANOVA Signup and view all the answers

What is the purpose of the F-statistic in ANOVA?

To determine the ratio of the model to its error Signup and view all the answers

What is the purpose of the sum of squares in ANOVA?

To partition the total variance into its components Signup and view all the answers

What is the main purpose of oblique rotation in factor analysis?

To allow for correlations between components Signup and view all the answers

What does the Kaiser-Meyer Olkin measure of sampling adequacy represent?

The proportion of variance that might be described by underlying factors Signup and view all the answers

What is the purpose of the pattern matrix in oblique rotation?

To provide factor loadings that are unique and exclude shared variance Signup and view all the answers

What is the assumption of Bartlett's test of sphericity?

That the correlation matrix is an identity matrix Signup and view all the answers

What is the minimum number of response options recommended for items in a scale?

3 Signup and view all the answers

What is the purpose of inspecting item distributions in strategy analysis?

To identify items with low correlations with other items Signup and view all the answers

What is the purpose of the anti-image correlation matrix?

To report the sampling adequacy of each item Signup and view all the answers

What is the interpretation of a Kaiser-Meyer Olkin measure of sampling adequacy of 0.8?

The solution is pretty good Signup and view all the answers

What is the consequence of having extreme scores in items?

It causes problems in the analysis Signup and view all the answers

What is the purpose of reporting the correlation between factors in oblique rotation?

To provide additional information about the relationships between the factors Signup and view all the answers

What is the primary purpose of reporting R2 change in a multiple regression analysis?

To explain the added variance in the dependent variable at each step Signup and view all the answers

In a statistical (step-wise) regression analysis, what determines the order of entry of predictors into the model?

The size of the correlations between the predictors and the dependent variable Signup and view all the answers

What is the main difference between a standard and a hierarchical regression model?

The importance of associations between predictors Signup and view all the answers

What is the term for the effect of a predictor on the dependent variable through a second predictor?

Indirect effect Signup and view all the answers

What is the condition necessary for a mediating variable to be considered a causal pathway?

The mediating variable must precede the dependent variable in time Signup and view all the answers

What is the purpose of a mediated regression analysis?

To test the causal effects of predictors on the dependent variable Signup and view all the answers

What is the term for the association between two variables that is due to a common cause?

Spurious effect Signup and view all the answers

What is the condition necessary for causation to be reported in a mediated regression analysis?

A known predictor and longitudinal research Signup and view all the answers

What is the purpose of the four steps in a classic mediation analysis (Baron & Kenny)?

To establish the causal relationships between the predictors and the dependent variable Signup and view all the answers

What is the main difference between cross-sectional and longitudinal research?

The ability to establish causation Signup and view all the answers

What is the primary aim of testing the significance of Path A in a mediation analysis?

To examine the relationship between the independent variable and the mediator Signup and view all the answers

In a mediation analysis, what is the interpretation of a non-significant Path c'?

Full mediation is not supported Signup and view all the answers

What is the purpose of factor rotation in principal component analysis?

To produce independent components Signup and view all the answers

What is the characteristic of an orthogonal rotation in principal component analysis?

Components are independent from each other Signup and view all the answers

What is the interpretation of a high factor loading on a particular component?

The item is strongly associated with the component Signup and view all the answers

What is the formula to calculate the total effect of the mediating pathway?

Path a * Path b Signup and view all the answers

What is the purpose of the Sobel Test in mediation analysis?

To test the significance of the mediating pathway Signup and view all the answers

What is the characteristic of a component matrix?

It represents the loading of items on each component Signup and view all the answers

What is the purpose of representing components in a two-dimensional space?

To visualize the loading of items on each component Signup and view all the answers

What is the consequence of not having an unbroken chain of events in the mediation model?

The mediation model is not valid Signup and view all the answers

What is the primary purpose of orthogonal rotation in factor analysis?

To produce a more meaningful representation of the data Signup and view all the answers

What is the difference between principal components analysis and factor analysis?

PCA is interested in all variance, while FA is only interested in shared variance Signup and view all the answers

What is the purpose of communality in factor analysis?

To determine the percentage of variance explained in an item by the factor solution Signup and view all the answers

What is the advantage of using varimax rotation in factor analysis?

It produces independent factors Signup and view all the answers

What is the purpose of reviewing the scree plot in factor analysis?

To determine the number of factors to extract Signup and view all the answers

What is the difference between a factor and a component in factor analysis?

A factor is used in FA, while a component is used in PCA Signup and view all the answers

What is the purpose of naming factors in factor analysis?

To describe the overall component that the factor represents Signup and view all the answers

What is the advantage of using a hierarchical approach in factor analysis?

It allows for the specification of a predicted structure Signup and view all the answers

What is the purpose of reviewing the factor matrix in factor analysis?

To determine the loading of each item on each factor Signup and view all the answers

What is the difference between a rotated and unrotated matrix in factor analysis?

A rotated matrix has been transformed to produce a more meaningful representation of the data Signup and view all the answers

Study Notes

Regression Basics

Standard multiple regression predicts a dependent variable (DV) using two or more independent variables (IVs) simultaneously.
IVs have equal importance to explanation.
Researchers not interested in associations between IVs.
Key terms:
- Variable: Measurable characteristic that varies (by groups, individuals, or time)
- Dependent/Outcome Variable (DV): Presumed effect in the analysis
- Independent/Explanatory Variable (IV): Presumed cause in an analysis
- Control Variable/Covariate: Variables that are not studied but included in the model/analysis
- Best Fitting Line: When plotting data, the most appropriate line showing the relationship between dependent and independent variables
- Residual: Deviators from the fitted line (estimated value) to the observed values (data point)
- Error: Difference between the observed value and the true value (often unobserved)
- Unique Variance: Variability in a DV uniquely explained by specific IV(s) in multiple regression, distinct from Pearson's, where unique variance isn't assessed
- Shared Variance: Variability in a DV, explained by multiple IV(s) simultaneously in both multiple regression and Pearson's correlation

Graphical Representation

Total Variance, explained and error

Regression Results

Regression Coefficient R2: represents the proportion of the variance in the dependent variable (the variable being predicted) that is explained by the independent variables (the predictors) in the model
Ranges from 0 (not explained) to 1 (explains all variability)
Unstandardized coefficient: the slope of the regression line reflecting the change in the DV from one-unit change in the IV, whilst holding all other variables constant (B)
Standardized coefficient: the slopes of the regression line expressed in standard deviation units (generally -1 to +1); making it comparable with other standardized coefficients
Semi-partial correlation (Part)sr2: Correlation between the predictor and outcome variable with variance shared between other predictors controlled in the predictor variable only
p-value of the model: It tests whether R2 is different from 0. A value less than 0.05 shows a statistically significant relationship

Hierarchical Regression Model

Hypothesis model – we determine what happens based on theory
Entered into model at different steps, based on theoretical importance or control
Associations between IVs important
Most theoretically important variables entered at different steps
Can test importance of different constructs

Statistical Regression Analysis (Stepwise)

M – not theory (not recommended) – based on the size of the correlations
Largest correlation is entered in first
Atheoretical (statistically driven)

Mediated Regression Analysis (Cue Ball)

Mediating variables theoretically explain how the predictor variables influence the DV (outcome)
The IV should precede the mediator in time, and mediator should precede the DV
Parallel mediator model: second mediator: can have two or more parallel mediators – need to be written for EACH indirect pathway association
Mediated Regression Analysis (Baron & Kenny)
4 Steps:
1. Path C: statistically significant association between IV and DV
2. Path A: statistically significant association between IV and mediator
3. Path B: statistically significant association between mediator and DV, after “controlling” for IV
4. Path c’: association of IV & DV, after controlling for mediator – should be non-significant (full mediation) or reduced (partial mediation)

Moderating Regression Analysis

Influence of one IV on DV “changes” based on score on second IV
The moderator variable is the IV that influences the relationship between IV and DV such as direction or strength
The IV is no longer independent; it is “conditional” on the moderator
Moderator is a “conditional effect” = b3 tell us the condition
Unconditional: The predictors each add variance to the explanation of the outcome variable, so each predictor is independent, so additive influence on the outcome
Moderator effects mean that the IVs are not independent
b3 = coefficient reflects the interaction between X*M eg. years in education * gender

ANOVA Basics

Are the means different?
Definitions and Terms:
- T statistics: Tests whether two group means are significantly different
- F statistics: the ratio of the model to its error
- Variability
- Between conditions: explained by our model
- Within conditions: unexplained error
- Sum of Squares
- SS Total: Grand Mean
- SS between: variance explained by our model
- SS within: variance not explained by our model
- Degrees of Freedom
- df for SS between: k-1 (number of conditions/groups minus 1)
- df for SS within: N-k (Number of participants minus number of groups)
- df SS total N-1 (number of participants minus 1)
- Omnibus test: tests for an overall experimental affect – that difference lies “somewhere”

Factorial ANOVA

Factorial Designs can show interactions
The impact of one independent variable (IV) ignoring the presence of any other IV included in the design
Main effect: influence of IV without regard for other IV’s in the analysis
Interaction: is the influence of one IV on score of DV conditional (dependent) on the other independent variable
One level depends on the other level
“The difference depends on..”

ANOVA Designs

Between groups: two experimental conditions and different people are assigned to each condition (drug trial)
AKA: “independent group”
Repeated measures: two experimental conditions and the same people take part in both conditions (can control if individual error – separate error terms)
Mixed ANOVA: combination of repeated and independent factors – participants 2 (reader group: dyslexia and control) × 2 (task difficulty: hard and easy): task difficulty repeated### Producing Independent Components
Initial component matrix produces a general component, but it's not a good way to separate independent components.
Factor rotation is used to reorganize the way variance is assigned to components, making them independent.
There are two types of factor rotation: orthogonal (independent) and non-orthogonal (correlated) rotations.

Component Factor Rotations

Orthogonal rotation:
- Variance is extracted from individual items.
- Components are independent from each other.
- Naming and describing components/factors.
- Interpreting outcome.

Representing 2 Components in 2D Space

Loadings for each component extend from 1 to -1.
Each item has a factor loading for each component, allowing representation in 2D space.
Components are perpendicular (90° angle) and independent from each other.

Orthogonal Rotation

Maintains independence of components.
Items stay in the same place in 2D space.
Principal axes are rotated to maximize the separation between the two groups.
Varimax rotation is an example of orthogonal rotation.
Variance accounted for in item-communality final (h2) is reported.

Naming Factors

If replicating a solution, use previous names.
Use a combination of items to describe the overall component.
Avoid using the name of a single item.

Summary of PCA/FA

PCA/FA are exploratory techniques that reduce a large number of items to smaller, more coherent dimensions.
PCA/FA are based on a correlation/covariance matrix.
Factor loadings are used to describe components descriptively.

Principal Components Analysis (PCA)

PCA is interested in finding what components have in common.
PCA explains the variance in each item using a few components.
Communality is the percentage of variance explained in an item by the factor solution.

Factor Analysis (FA)

FA is interested in finding the underlying components of a construct.
FA is more theoretically driven.
FA only uses shared variance (co-variance) between items.

Assumptions of Analysis

Bartlett's test of sphericity determines if there are factors/components in the correlation matrix.
Kaiser-Meyer Olkin measure of sampling adequacy describes the proportion of variance that might be described by underlying factors.

Distribution of Items

Scales should be suitable for PCA/FA.
Non-discriminating items (same score) and extreme scores can cause problems.
Items should have a minimum of 3 response options, with 4 or more being better.

Strategy Analysis

Inspect item distributions.
Correlation matrix: exclude items with correlations < 0.3 with at least one other item.
Assess sampling adequacy using the Kaiser-Meyer Olkin measure.

Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

Description

This quiz covers the fundamentals of regression analysis, including definitions and terms such as dependent and independent variables, and how they are used to predict outcomes.