RD #5: PART 2

• Statistical significance is a factor in research
• Quasi-experimental designs exist
• Multivariate designs exist
• Paired t-tests are used
• ANOVA (1 vs 2 way) statistical test can be used
• Post-hoc tests are available
• Baselines indicate amount of variability in control conditions
• Replication increases power and validity
• Replication may occur across different behaviors, environments, and over time

Single-Subject Design Analysis

• Single-subject design analysis involves separate lines for the baseline and treatment, then a comparison of slopes
• Alternatively, computes a regression line for each participant
• Single-subject design analysis compares level, trend, slope and variability

Two Standard Deviation Band Method

• Calculate the mean and standard deviation for the baseline data
• Then draw a baseline line +/- 2 SD and extend this line through the treatment phase
• Then examine how many data points fall outside the SD range

Percentage of Non-Overlapping Data

• Range is determined in each phase
• Percentage of data points is determined that fall within in overlapping ranges

Single-Subject Designs: Social Validation

• Establishing the importance of the treatment effect
• Also, assesses the acceptability of treatment procedures regarding patient preference, comfort, safety, cost, and practicality
• Determines the social importance of target behavior and magnitude treatment effects
• Can evaluate if treatments made functional change or not beyond statistical significance

Analysis of Relationships

• Uses correlations to look at the linear relationship between 2 variables

Correlations

• Based on covariance, 2 variables will vary in similar patterns
• Partial correlations find the relationships between 2 variables while holding effects of a 3rd variable constant, effectively removing its effect

Correlations Assumptions

• Assumes a linear relationship
• Requires adequate variability in both scores to get a correlation
• Should be no floor or ceiling effects because otherwise, correlations will be low
• Homoscedasticity assumes homogeneity of variance but multivariate such that variables have equal variability
• This results in participant and control groups vary to the same degree

Interpreting Relationships

• Involves direction,strength, variance shared, significance and confidence intervals

Interpreting Relationship - Scattergram

• Shows the relationship between two variables
• Plotted using dots
• A line of best fit can be used for the scattergram
• Line of best best is straight and unique to the dataset
• This is done by minimizing the value from summing the distance of each point from the line

Direction of Relationship

• Positive relationships mean both variables increase or decrease in the same direction, with a slope up to the right
• Negative relationships mean variables move in opposite directions, with a slope down to the right

Strength of Relationship

• Determined visually by how close the dots are to the line of best fit
• Correlation coefficients exist ('r') to determine strength of relationships
• Note strength is measured by statement of strength of relationship
• Ranges from between 0.00 & ±1.00
• 0.00 - 0.25 is little or no relationship
• 0.25 - 0.49 is a fair relationship
• 0.50 - 0.69 is a moderate relationship
• 0.70 - 0.89 is a high relationship
• 0.90 - 1.00 is a very high relationship
• Strict cut-offs do not exist
• Correlation is affected by sample size, measuring error and type of variables
• Meaningfulness depends on context and how closely related variables are

Variance Shared

• Looks at practical/clinical significance by measuring how much variance is accounted for
• Coefficient of determination represents r squared
• Effect sizes are based on variance, such eta square & omega squared

Significance of Relationship

• Uses statistical testing of strength in all correlations
• Must test the null-hypothesis, H0, that there is no relationship exists between the variables, and that r = 0
• Statistical significance is reached by ensuring the p value is at or below alpha level
• Hypothesis can be rejected and it can be concluded that two variables are correlated

Confidence Intervals

• Represents range in which "true" score lies
• Set to 95% usually
• Uses a range of given scores
• Larger sample sizes provides a smaller confidence interval

Correlations - Parametric

• Uses Pearson Product Moment Correlation - r
• Uses same assumptions as other parametric tests

Correlation coefficient types

• Correlation coefficient is a stat representing relationships between 2 or more things
• Pearson product moment correlation (r) is for continuous numbers and compares two items
• Intraclass correlation is for continuous numbers and compares two or more items
• Spearman rank order correlation (rs) is for ordinal (ranked) numbers and compares two items
• Kendall's tau (τ) is for ordinal (ranked) numbers and compares two items
• Cohen's kappa (к) is for nominal numbers and compares two items but can be adapted for more
• Phi coefficient (Ф) is for nominal numbers and compares two items
• Cramer's V is for nominal numbers and compares two items
• Biserial correlation (rb) is for interval & ordinal numbers and compares two items
• Point-biserial correlation is for interval & nominal numbers and compares two items

Correlations Interpretations

• Interpreting correlation coefficients must acknowledge correlation ≠ causation
• Observed relationships may be caused by intermediary variables
• Correlations might imply causality when logical time sequences are identified
• More indications are plausible biological explanations, a dose-response relationship and consistency of findings across studies

Correlation Interpretation - Generalization

• Generalizations must be within tested range
• Impossible to know what would happen before or after
• Restricted range of scores might not reflect true relationship
• Therefore measure over full range

Linear Regression

• Assumes a linear relationship between the IV and DV
• Seeks to predict DV from IV
• The IV is sometimes called predictor variable
• There are Bivariate and Multivariate types

Bivariate Regression

• Uses the correlation between one independent predictor variable (X) and one dependent variable (Y) to predict Y, e.g. reading predicted by phonemic awareness
• Calculates a "line of best fit" known as a regression line
• Line yields the smallest residuals from each participant or data point (i.e., their delta)
• The equation of any sample regression line is an approximation of the population regression line

Bivariate Regression Data

• Actual Y and the predicted values of Y will differ for any data set
• The distance from the regression line of actual values of Y is called residuals
• Intercept and slope allows to predict an individual's score, e.g. predictive reading score
• This is done using an equations, where Ŷ = a + bX
• Ŷ = predicted value of Y
• a = Y intercept, e.g. Value of Y when X = 0
• b = slope of regression line
• X = value of independent variable

Bivariate Regression - Outliers

• Bivariate Regression is susceptible to outliers
• Therefore outliers should be omitted
• Or do a comparative analysis with and without outlier to estimate its effect

Bivariate Regression Accuracy

• Accuracy of prediction is measured by the coefficient of Determination = r2
• It represents an effect and the variation in the dependent measure
• Regression is accurate when the multiple coefficient of Determination is higher
• Example, when r = .50, r2 = .25 meaning that 25% of the variance in the dependent measure can be explained by the predictor variable
• Also requires evaluating accuracy against a a Beta weight which measures regression coefficient - e.g. Y' = a + b₁X1, indicating how much variable is contributing

Regression ANOVA

• Tests observed relation between X and Y to see if it happened by chance
• This is done using F test/p value in ANOVA

Advanced Procedures

• These advanced procedures exist: reliability analysis, multiple regression analysis, canonical correlation analysis, discriminant analysis and factor analysis

Reliability Analysis

• Reliability uses Pearson Product Moment Correlation Extensions -Paired t-test, slope & intercept documentation and SEM & confidence interval
• Uses intraclass Correlation Coefficients, and Two or more continuous variables
• Kappa also, using two or more nominal variables with 2 or more categories, and accounts for 'chance' using scales from .00-1.00

Multiple Regression Analyses

• This approach uses more than one IVs to predict single DVs
• Equation, Y' = a + β₁X₁ + B2X2 + B3X3 ...
• Goal is best possible predicition
• R = the correlation coefficient of multiple IVs
• amount of variance explained by the model
• Can be concurrent or predictive
• The model accounts for the predictive power
• Examples are; predicting reading scores or hearing aid compliance

Multiple Regression Analysis - Assumptions

• Requires linear relationship of predictor to DV
• Requires homogeneity outliers must be removed via data transformations
• Requires no Multicollinearity so reduce correlations amongst each other
• Select on variable, 10 participants are required per variable

Multiple Regression Analysis - Types

• Simultaneous method adds the IVs all at once
• Stepwise adds one at a time based on which adds most variance through the Pearson correlation coefficient
• Hierarchical is used to control the order of addition based on theory

Multiple Regression Analysis - Beta weights

• Beta weights are Standaized regression coefficients, e.g. Y = a + ẞ₁X₁ + B2X2 + B3X3
• Tests occur to to observe relations happened by chance via a p-val from ANOVA
• Get each R2, and R² change

Multivariate Analyses

• 1 IVs predicting multiple DVs taken as a group

• Example DVs like word decoding, reading fluency and reading comprehension,
• Also, IVs like PA, vocabulary and MLU
• Logistic Regression is used for predicting dichotomous outcomes
• A typical DV would wear hearing aids vs in drawer, versus hearing aid improvement

Multivariate Analyses - Discriminant Analysis

• 1 IVs predict a group membership (DV)

• This obtains equations needed for group prediction, while evaluating +ves & -ves
• Can be descriptive and looking to see what variables differentiate given partipants
• Also, predictive (prescriptive) and assigns individuals not diagnosed

Multivariate Analyses - Factor Analysis

• Seeks to factors among variable in predicting a DV
• Analyzes sub components
• Determines Eigenvalues via variability percentages
• Typically eigenvalue ≥ 1 is important
• Factor loadings identify each factor