PSYC2018H F24 Lecture 9: Survey Methods PDF

# Chapter 9: Non-Experimental Design I: Survey Methods ## Chapter Objectives - Explain why sampling issues are more relevant for survey research than for most other research in psychology. - Articulate the principles of good survey construction. - Explain the problems that can make it difficult t...

# Chapter 9: Non-Experimental Design I: Survey Methods ## Chapter Objectives - Explain why sampling issues are more relevant for survey research than for most other research in psychology. - Articulate the principles of good survey construction. - Explain the problems that can make it difficult to interpret survey data. - Describe four ways to collect survey data and list the advantages and disadvantages of each. - Describe three varieties of probability sampling and know when each is used. - Distinguish between positive and negative bivariate correlations, create scatterplots to illustrate them. - Calculate a coefficient of determination ($r^2$) and interpret its meaning. - Understand how a regression analysis accomplishes the goal of prediction, and distinguish between simple linear regression and multiple regression techniques. - Understand how directionality can make it difficult to interpret correlations and how a cross-lagged panel study can help with the directionality problem. - Understand the third variable problem and how such variables can be evaluated and controlled through a partial correlation procedure. - Distinguish between mediators and moderators within the context of understanding third variables in a correlation. ## Survey Research - **Learning From History**: - Darwin (facial expressions of emotion): - leading, overly specific - Galton (origins of scientific interests): - "huh" type questions: "How far do your scientific tastes appear to be innate?" - Hall (contents of children's minds): - methodological issues - Titchener & James: - flames for the pests - **Sampling issues in survey research:** - biased vs. representative samples: - Non-probability vs. Probability sampling (see Ch. 4) - Self selection bias: - election of 1936 (Literary Digest example) - subscribers + phones and cars - demand characteristics? - **Surveys < Psychological Assessment**: - Attitudes, opinions, beliefs, projected behaviors VS. Psychological functioning ## Types of Survey Questions - **open-ended vs. closed questions:** - "Do you like this example?" - **the "most important problem" question** - **Use of Likert scales:** - avoid response bias ("like"ert scales) - **Assessing memory and knowledge:** - Moderate use of DK alternative - **Adding demographic information:** - Basic identifying data (e.g., age, income) - Place at end of survey ## Creating an Effective Survey - **Survey wording:** - avoid ambiguity (pilot study helps): - "Research methods can be fun." - Don't ask for two things in one question: - "Do you want America to be great again by getting rid of all the environment? in this country and saving the" - Avoid biased and leading questions: - "Do you beat your students less often?" ## Collecting Survey Data - **In-person interview surveys (e.g., Kinsey):** - Plus in-person, comprehensive, follow-ups possible - Minus representative samples, cost, logistics, interviewer bias - **Mailed written surveys:** - Plus? in-person, ease of scoring - Minus cost, response rate (nonresponse bias), social desirability bias - **Phone surveys:** - Plus cost, efficiency - Minus must be brief, response rate, SUGging - **Electronic surveys:** - Plus cost, efficiency - Minus sampling issues, ethics ## Analyzing Data from Non-Experimental Designs - **Correlation: Describing relationships:** - Finding the relationship between two variables without being able to infer causal relationships - Correlation is a statistical technique used to determine the degree to which two variables are related - Three types of \[linear] correlations: - Positive correlation - Negative correlation - No correlation - **Positive correlation:** - Higher scores on one variable associated with higher scores on a second variable - **What is the relationship between the number of hours spent per week studying and GPA?** - | Student | Study Hours | GPA | - |---|---|---| - | 1 | 43 | 3.3 | - | 2 | 23 | 2.9 | - | 3 | 31 | 3.2 | - | 4 | 35 | 3.2 | - | 5 | 16 | 1.9 | - | 6 | 26 | 2.4 | - | 7 | 39 | 3.7 | - | 8 | 19 | 2.5 | - **Negative correlation:** - Higher scores on one variable associated with lower scores on a second variable - **What is the relationship between the number of goof-off hours spent per week and GPA?** - | Student | Goof-Off Hours | GPA | - |---|---|---| - | 1 | 42 | 1.8 | - | 2 | 23 | 3.0 | - | 3 | 31 | 2.2 | - | 4 | 35 | 2.9 | - | 5 | 16 | 3.7 | - | 6 | 26 | 3.0 | - | 7 | 39 | 2.4 | - | 8 | 19 | 3.4 | - **Scatterplots:** - Graphic representations of data from your two variables - One variable on X-axis, one on Y-axis - Fig. 9.1: - (a) r = +1.00 - (b) r = -1.00 - (c) r = 0 - (d) r = weak positive - (e) r = weak negative - (f) r = strong positive - (g) r = strong negative - **Creating a scatterplot from data:** - Each point represents an individual subject - Fig. 9.2 - **Scatterplots from the hypothetical GPA data for positive (top) and negative (bottom) correlations:** - Fig. 9.3: - (a) r = +0.88 - (b) r = -0.89 - **Correlation coefficients:** - Statistical tests include: - Pearson's r, Spearman's rho, phi coefficient - Ranges from -1.00 to +1.00 - Numerical value ? strength of correlation: Closer to -1.00 or +1.00, the stronger the correlation - Sign ? direction of correlation: Positive or Negative - **Coefficient of determination:** - Equals value of Pearson's $r^2$ - Proportion of variability in one variable that can be accounted for (or explained) by variability in the other variable. - The remaining proportion can be explained by factors other than your variables. - r ? $r^2 = .36$: 36% of the variability of one variable can be explained by the other variable. 64% of the variability can be explained by other factors. ## Assignment to Test Correlation - Pearson's Correlations - Variable: Test - Assignment: - Pearson's r = 0.409 - p-value < .001 ## Analyzing Data from Non-Experimental Designs - **DesignSof outliers:** - Scores dramatically different from remaining scores in data set impact Pearson's r and $r^2$ - Could lead researcher to making Type I error - **Regression: Making predictions:** - The process of predicting individual scores AND estimating the accuracy of those predictions - Regression line - straight line on a scatterplot that best summarizes a correlation - $Y = a + bX$: - Y = criterion variable—the variable that is being predicted: Predicting GPA from study hours ? Y = GPA - X = predictor variable—the variable doing the predicting Predicting GPA from study hours ? X = study hours - a = point where regression line crosses Y axis - b = the slope of the line - Use the predictor variable (X) to predict the criterion variable (Y) - **Regression lines for the GPA scatterplots:** - Study time (X) of 40 predicts GPA (Y) of 3.5 - Goof-off time (X) of 40 predicts GPA (Y) of 2.1 - Fig. 9.4 - **Regression: Making predictions:** - Research Example 26 - Linear regression: - Predictor variable: passion for studying - Criterion variables: academic engagement and burnout - Multiple regression: - Predictor variables: passion for studying AND motivation - Criterion variables: academic engagement and burnout - **Other concepts: survey data, shared variance ($r^2$), applied research, external validity, "what's next?"** ## Interpreting correlational results - **Directionality problem:** - Given correlation between A and B - A could cause B, or B could cause A - **Research Example 27: Watching TV and aggression:** - Cross-lagged panel correlation (Fig. 9.5) - If B (aggressiveness) happens ten years after A (preference for violent TV), A could cause B, but B cannot - **Mediating vs. Moderating variables:** - Either can help explain a correlation. - Mediator: explains how or why a relationship b/t two variables exists. - Moderator: explains under what conditions the relationship b/t two variables exist. - Fig. 9.6 ## Summary - Surveys are used to obtain information primarily about attitudes, beliefs, opinions and/or projected behaviors. - Researchers should have clear empirical questions and take care in survey working. - Correlation and regression are statistical techniques that can be used to analyze data from non-experimental designs, including survey data. - Always be aware of the directionality problem and potential third variables when interpreting correlational research. - Non-experimental research is used in a variety of ways, and can serve as an avenue to explore related ideas with an experimental approach.