Lesson 4 - Survey Research PDF

Hi Students! Thank you for joining our session today. We'll start in a while  Experimental Psychology PLMUN DEPARTMENT OF PSYCHOLOGY MR. BENJAMIN HERNANDEZ  Mental Health Check-in Use the Google Meet reactions to share how are you feeling today...

Hi Students! Thank you for joining our session today. We'll start in a while  Experimental Psychology PLMUN DEPARTMENT OF PSYCHOLOGY MR. BENJAMIN HERNANDEZ  Mental Health Check-in Use the Google Meet reactions to share how are you feeling today Really great! Starting to struggle Pretty good having a hard time okay, I guess I need support Alternatives to Experimentation: Survey and Interviews Objective Objective Objective Objective Learn the pros and Learn how to Learn the factors Learn about more cons of different administer involved in designing nonexperimental sampling techniques questionnaires and questionnaires and techniques for studying conduct interviews devising good behavior: survey and questions interview research Survey research Survey research obtains data about opinions, attitudes, preferences, and behaviors using questionnaires or interviews. The survey approach allows researchers to study private experience, which cannot be directly observed What are the advantages of the survey approach? We can efficiently collect large amounts of data. Anonymous surveys can increase the accuracy of answers to sensitive questions Surveys can allow us to draw inferences about the causes of behavior and can complement laboratory and field experiments. What is the most important limitation of the survey approach? The survey approach does not allow us to test hypotheses about causal relationships because we do not manipulate independent variables and control extraneous variables. Therefore surveys are low in manipulation of antecedents. However, they can be low or high in imposition of units Survey are usually written or face to face Constructing Surveys What are the major steps in constructing surveys? 1. Identify specific research objectives. If you want to measure peoples’ beliefs about animal research, you need to figure out which specific part of animal research you wish to ask about. 2. Decide on the degree of imposition of units (degree of response restriction). Do you want low (can say or write they want) or high (must answer yes or no) imposition of units? 3. Decide how you will analyze the survey data. This depends upon the imposition of units. High imposition needs statistics, low uses qualitative methods. Describe the major question types. Closed questions (structured questions) can be answered using a limited number of alternatives and have a high imposition of units. For example, “how do you feel about the job our president is doing? Answer either very good, good, fair, or poor.” Describe the major question types. Open-ended questions (open questions) require that participants respond with more than a yes or 1-10 rating and have a low imposition of units. For example, “Why did you choose your major?” Difficulty with questions Many questions on surveys have problems in how they were written. They can be ambiguous, too complex or double barreled. Ambiguous - people don’t understand what the question is asking. For instance, the question, Have you ever considered the idea of abortion? Too complex - use a double negative in a sentence or a word whose meaning is difficult to understand. “I don’t usually dislike not being alone.” Double barreled - two ideas are present in the question. Do you feel the country is going in the right direction and the president is doing a good job? How do researchers analyze data from each question type? The number or percent of responses can be reported for closed questions. Open-ended questions can be analyzed using content analysis, like Yepez’s INTERSECT, in which responses are assigned to categories using objective rules. Measuring Responses Nominal Scale Simplest level of measurement is a nominal scale. A nominal scale assigns items to two or more distinct categories that can be named using a shared feature, but does not measure their magnitude. True, false, male, female For example, you can sort professors into exciting and dull categories Ordinal Scale An ordinal scale measures the magnitude of the dependent variable using ranks, but does not assign precise values. For example, ask a subject to list his favorite soda from favorite to lease favorite. You don’t really know how much he likes his third ranked soda. Interval Scale An interval scale measures the magnitude of the DV using equal intervals between values with no absolute zero point. For example, Fahrenheit or Centigrade temperatures, and Sarnoff and Zimbardo’s (1961) 0-100 scale. Zero temperature is not a true zero, true zero is when there is a total absence of something. How much do you like this professor, 0 = not at all, 1 = very little, 2= a little, 3= like him, 4 = like him a lot Ratio Scale A Ratio Scale measures the magnitude of the dependent variable using equal intervals between values and an absolute zero. This scale allows us to state that a 2-meter board is twice as long as a 1-meter board. For example, distance in meters, time in seconds Which to choose? Ordinal gives you more information than nominal. So knowing which candidate came in first, second etc (ordinal) is more informative than just knowing who won and who lost (nominal). But get more information knowing percentage of people who voted for each (ratio scale). How should we select measurement scales? The best type of scale depends on the variable you are studying and the level of precision you desire. Marital status would be nominal, years married would be ratio. Since psychological variables like traits, attitudes, and preferences represent a continuous dimension. Each individual can fall at any point along the dimension, such as high sociability or low sociability. Different scales can be used to measure continuous dimensions including interval, or ratio. How should we select measurement scales? When working with variables like sociability, psychologists often select the highest scale possible since it provides more information and allows analysis using more powerful statistics. Important consideration for Survey Items Subjects decide to refuse to answer surveys during the start or first few questions Engage subjects from the start by asking interesting questions they will not mind answering. Important consideration for Survey Items The first survey question should be: 1. Relevant to the survey’s central topic 2. Easy to answer 3. Interesting 4. Answerable by most respondents 5. Closed format (so they can’t say “I don’t know” What is a response style? Response styles are tendencies to respond to questions or test items without regard to their actual wording. People differ in their willingness to answer, position preference, and yea-saying and nay-saying Willingness to answer Is the tendency to guess or omit items when unsure. If you tell subjects that there are no right answers may be more responses. Position preference Is selecting answer based on its position. For example, students choosing “c” on multiple-choice exams. Or consistently answering “true” without reading the question because all the other statements were “true” Manifest Content Is the plain meaning of the words printed on the page. While we expect subjects to respond to the manifest content of questionnaires, they may ignore it when answering questions about their feelings or attitudes. Subjects may just say “yes” to everything rather than pay attention to the question Yea-saying and nay-saying? Yes-saying is agreeing with an item regardless of its manifest content Nay-saying is disagreeing with an item regardless of its manifest content. To control for this, must mix up questions with some needed a “no” response and some needing a “yes” response. Example: 1. I am happy most of the time 2. I enjoy being with other people 3. When I can, I avoid noisy places. Context effects Context effects are changes in question interpretation due to their position within a survey This problem is especially likely when two questions are related and not separated by buffer items (unrelated questions) Example of context effects You want people to rate Miley Cyrus on this scale: Nice -------------- Nasty If this item appeared right under another item asking the person to rate Kerry Washington as: Not sexy -------------- Sexy The subject might rate Miley Cyrus according to the “sexy” question that appeared before it. To control for this you put a buffer question in between, like, Are the Koch brothers: Kind --------------- Cruel Collecting Survey Data Social Desirability Response Set The social desirability response set is representing ourselves in a socially appropriate fashion when responding to a question’s latent content (underlying meaning). It is when a subject responds to a question in a way to make himself look good in the experimenter’s eyes. for example, a question may ask whether you are able to get along with loud-mouthed obnoxious people. The person responds “yes” to be more likeable Social Desirability Scale There are scales available to show if a person has high social desirability. If he does, you may not be able to use his data in your analyses. For instance, the Marlow-Crowne Social Desirability Scale. https://www.cengage.com/resource_uploads/downloads/04 95092746_63626.pdf https://www.carepatron.com/files/marlowe-crown-social- desirability-scale.pdf Collecting Survey Data Written questionnaires Mail surveys Telephone surveys Internet surveys Interviews Focus groups Written questionnaires Keep instructions simple and clear Control for reactivity by giving privacy Keep anonymous If questions can be embarrassing, be prepared to minimize discomfort by giving privacy, not looking at their responses, assuring anonymity Mail Always include a cover letter that is polite Include a stamped envelope for return Response rates are between 45% to 75% If possible, include a small gift Keep track of those who do not return it; consider a second mailing. When nonreturn rate is high, this compromises your results (perhaps only those interested in the topic returned it, volunteer bias) Computer and Internet Can use programs such as surveymonkey.com and surveygizmo.com Allows for easier collection and analyses Less concern with social desirability, feels more anonymous Allow for larger pool of collection data Cannot tell if person takes survey multiple times Telephone Large scale telephone surveys use random digit dialing, not a phone book to make calls. Allows for a wider sample Response rate for phone surveys is 60 to 90% Caller ID allows more people to refuse to answer call. Male telephone interviewers are more effective than females. Interviews Most expensive and time consuming method Female interviewers tend to be more successful than male Must be able to establish a rapport Best results come when interviewer matches the respondent on race, physical appearance, socioeconomic status. Will interview be structured or unstructured Structured and Unstructured interviews In structured interviews, questions are asked the same way each time. It is read from a script. This provides more usable, quantifiable data. In unstructured interviews, the interviewer can explore interesting topics as they arise. These data may not be usable for content analysis. Focus Groups Small groups of people with similar characteristics, all women, or all teachers. The interviewer is called the facilitator Facilitator guides the group through a discussion of specific issues. Are usually open-ended questions Evaluating Validity - Does the survey measure what it’s supposed to measure? Sampling Sample and Population Sampling is deciding who will fill out your survey. A population consists of all people, animals, or objects that share at least one characteristic. For instance, all undergraduate students, all senior citizens. A sample is a subset of the population of interest. Data collected from the sample can be used to draw inferences about the population. Generalizability How accurately we can generalize our findings from a given sample to a population depends upon its representativeness. Representativeness is how closely the sample mirrors the larger population. How closely the sample responses reflect what we would obtain if we could sample the entire population. There are 2 sampling approaches, probability sampling and nonprobability sampling Probability Sampling Probability sampling- selecting subjects in such a way that the odds of their being in the study are known or can be calculated. If target population is all undergraduate students in city college, we could get a count from city records, then we would know the odds of any one person being in the study. Researcher must use an unbiased method of choosing the participants, such as flipping a count, taking names out of a hat, using a table of random numbers. This is called random selection. Probability Sampling advantage 1. A probability sample is more likely to represent the population (external validity) than a nonprobability sample. 2. We know the exact odds of members of the population being included in our sample. This tells us whom the sample represents. Main Probability Sampling Methods The four main probability sampling methods are: simple random sampling – a portion of the whole sample is selected in a random way (close your eyes and pick out 20 names from all the names if need 20 people in your study) systematic random sampling – list all the people in the population in unbiased way and take every nth one Main Probability Sampling Methods stratified random sampling- randomly sampling from people in each subgroup in the same proportions as they exist in the population. If school is made up of 70% female and 30 % male, then want your sample to also be 70% female and 30% male. cluster sampling – sample entire clusters or naturally occurring groups that exist in the population. Randomly select from clusters that already exist, such as zip codes and survey everyone in that zip code. Only works if the cluster is similar to the rest of the population. Used when the population is very large. Nonprobability sampling Probability sampling is best, but can’t always be used. Most studies are actually done using nonprobability sampling. Here subjects are not chosen at random. Main Nonprobability sampling The four main nonprobability sampling methods include: quota sampling- Researcher has quotas she must fill for her data, needs 50 white and black men to answer questions. Doesn’t matter how they are selected, as long as you fill quota. Goes to Iona College and gets first 50 white and black men that she sees to fill out her surveys. convenience sampling – using groups that happen to be available, your class, your choir group. Is commonly used. Aslo called accidental sampling. Main Nonprobability sampling purposive sampling – selecting a sample who are needed for the purpose of a study. If purpose of study is to compare freshman to seniors on psychology comprehension, then this is purposive sampling. snowball sampling- researcher locates a few people who fit the sample criterion and asks them to locate additional individuals. If want to sample men who are sports enthusiasts, then you locate a few, an ask them to give the survey to some of their friends who are also sports enthusiasts. That’s it! Time for some Q&A, any question?