Chapter 4_ Alternatives to Experimentation_ Surveys & Interviews Reviewer.pdf

UNIVERSITY OF MAKATI Institute of Psychology J.P. Rizal Ext. West Rembo, Makati City REVIEWER CHAPTER 4: Alternatives to Experimentation: Surveys & Interviews is a use...

UNIVERSITY OF MAKATI Institute of Psychology J.P. Rizal Ext. West Rembo, Makati City REVIEWER CHAPTER 4: Alternatives to Experimentation: Surveys & Interviews is a useful way of obtaining information about people’s opinions, attitudes, preferences, and behaviors simply by asking. Gather large amounts of data efficiently. Advantages of Surveys Efficiency: Surveys allow for the collection of large amounts of data from a large number of participants in a relatively short period of time. Versatility: Surveys can be used to gather information on a wide range of topics, from opinions and attitudes to behaviors and experiences. Anonymity: Surveys can provide anonymity, which can encourage honest responses, especially on sensitive topics. Cost-effective: Surveys can be relatively inexpensive to conduct, especially compared to other research methods such as experiments or in-depth interviews. Generalizability: Surveys can be used to make inferences about a larger population based on the responses of a sample. Disadvantages of Surveys Self-report bias: Respondents may provide inaccurate or biased answers due to social desirability bias, response set bias, or other factors. Limited depth: Surveys may not provide in-depth information about participants' experiences or motivations. Low response rates: Surveys can have low response rates, which can limit the generalizability of the findings. ○ Time constraint ○ Lack of interest ○ burden Question bias: Poorly worded questions can lead to biased or misleading results. Lack of control: Surveys do not allow for the manipulation of variables, making it difficult to establish causal relationships. ○ lack of control over the independent variable; cannot be manipulated.  Constructing a good survey is a research project in itself, with specific steps to follow to ensure valid information is gathered and that the questions truly address the researcher's objectives. Things to remember when constructing a survey: 1. Start your survey by clearly outlining your specific research objectives. a. Target audience b. Type of survey 2. Design your survey questions. Decide how you are going to address the imposition of units. a. Closed-ended questions Also called structured questions They are answered by one of a limited number of alternatives from which a participant must choose. Take note! Closed-ended items are used when researchers have a good idea of the different responses that participants might make. Closed-ended items are more difficult to write because they must include an appropriate set of response options. However, they are relatively quick and easy for participants to complete. They are also much easier for researchers to analyze because the responses can be easily converted to numbers and entered into a spreadsheet.  B. Open-ended questions Also called open questions Solicit information about opinions and feelings by asking questions so that the person must respond with more than a yes, no, or 1-10 rating. Take note! Open-ended items are useful when researchers do not know how participants might respond. Open-ended items are relatively easy to write because there are no response options to worry about. However, they take more time and effort on the part of participants, and they make it more difficult for the researcher to analyze the answers or responses. ○ It can often be used to clarify or expand answers to closed questions. ○ Content analysis - to quantify answers to open questions, responses need to be assigned to categories that are created from the data according to objective rules or guidelines. Content Analysis 3. Avoid double-barreled (compound) questions Asks for responses about two (or more) different ideas in the same question. For example: Do you like cats and dogs? Do you like men and women? Take note! Response choices need to be exhaustive, meaning they need to contain all possible options. However, only use this latter option if you do not expect people to use it very often because it is difficult to interpret or analyze an option that could contain many different answers.  a. Levels of Measurements b. Selecting Levels of Measurement Take note! Different format of closed-ended survey questions = Different levels of measurement Different levels of measurement = Different statistical tests A. Levels of Measurement 1. Nominal Scale- simplest level of measurement Lowest level of measurement. 2. Ordinal Scales - is a rank ordering of response items. 3. Interval Scales - Numbers with equal intervals but no true zero point. 4. Ratio Scales- the highest level of measurement. It has equal intervals between all values and a true zero point. B. Selecting Levels of Measurement The format of your closed-ended survey questions can significantly impact the statistical analysis you can perform on the collected data. Different question formats correspond to different levels of measurement, which in turn determine the appropriate statistical methods to use. Survey responses can be measured by using one of the four types of scales It is possible to measure a response by more than one of these scales Unipolar scale - a scale where only one construct is tested such as frequency (Never, Rarely, Sometimes, Often, Always) Bipolar scale - a scale where there is a dichotomous spectrum (e.g. Completely dissatisfied, Mostly dissatisfied, Somewhat dissatisfied, Neither satisfied nor dissatisfied, Somewhat satisfied, Mostly satisfied, and Completely satisfied) The level of measurement dictates the appropriate statistical methods: 1. Nominal Scale: Frequency distributions, mode, chi-square tests. 2. Ordinal Scale: Frequency distributions, mode, median, rank-order correlations (e.g., Spearman's rho).  3. Interval Scale: Frequency distributions, mode, median, mean, standard deviation, t-tests, ANOVA, correlation (e.g., Pearson's r). 4. Ratio Scale: Frequency distributions, mode, median, mean, standard deviation, t-tests, ANOVA, correlation (e.g., Pearson's r), regression analysis. Two primary factors influencing this choice : 1. The nature of the variable you are studying. 2. How much measurement precision you desire: Higher levels of measurement (interval or ratio) Lower levels of measurement (nominal or ordinal) Researchers generally use the following rule for selecting the right level of measurement for multiple levels: When different levels of measurement will “fit” equally well, choose the highest level possible because it provides more information about a response. Ratio and interval scales tend to be preferred by researchers over ordinal and nominal measurements. - Response Styles According to Czaja and Blair (1996), in addition to being the start of data collection, the first question should have these characteristics: 1. Relevant to the central topic 2. Easy to answer 3. Interesting 4. Answerable by most respondents 5. Closed format (1) Subjects do not have to think about a great deal (no open-ended questions) (2) Subjects are able to answer without saying “I don’t know” and; (3) Subjects will think are relevant to the topic of the survey. (4) Make sure your questions are not value-laden. Do not word your questions in ways that would make a positive or negative response seem embarrassing or undesirable. Response Styles are tendencies to respond to questions or test items in specific ways, regardless of the content. 1. Willingness to answer - comes into play whenever questions require specific knowledge about facts or issues. (a) Some researchers attempt to control for this factor by explicitly telling subjects to guess if they are not sure of the answer to a question. (b) Or you might tell subjects that there are no right or wrong answers to the questions (if that is true); sometimes that helps. 2. Position preference - when in doubt about answers to multiple-choice questions, some people always select a response in a certain position, such as answer c. (a) In a multiple-choice survey gathering data about attitudes toward abortion, for instance, do not always put “prochoice” responses as option c. 3. Yea-sayers (response acquiescence) - people who are apt to agree with a question regardless of its manifest content. 4. Nay-sayers (response deviation) - tend to disagree no matter what they are asked. Manifest content - the plain meaning of Manifest content is the words or questions that actually appear on the page. - “Have you ever visited another country?” means just that; the manifest content of the item is simply foreign travel. When we phrase questions to have simple yes/no or agree/disagree answers, we make it easy for subjects to respond based on response style. By building some specific content into the options as in item 2, we encourage subjects to give more thought to each choice.  - Remember this: In version A, all items are designed so that saying yes leads to a high unfounded optimism score. This means that people who tend to say yes a lot could score high on unfounded optimism, even if they are somewhat pessimistic. - Now, consider version B. To score high on unfounded optimism in version B, a person would have to give both yes and no answers. - Using version B would result in more accurate answers because people who score high on unfounded optimism are probably not the type to say yes to everything. Pre-test your questions Pre-testing can catch context effects; sometimes the position of a question can influence how the question is interpreted. Context effects - effects produced by the position of a question; where it falls within the question order can influence how the question is interpreted. Ethical Considerations The potential for ethical problems needs to be just as carefully considered in survey research as in any other kind of psychological research. ○ Confidentiality of respondents ○ Potential reaction to answering survey questions. 1. Self-Administered Questionnaires ○ The instructions are simple and clear Avoid jargon or overly complex language. ○ Consider administering questionnaires privately whenever possible. consider the possibility of reactivity the potential for subjects to alter their responses due to the presence of an observer. ○ Weigh the pros and cons of group sessions.  While group sessions can be convenient for collecting data from multiple individuals at once, they may not be as conducive to serious responses, especially for sensitive topics. Consider the potential for embarrassment or distraction within a group setting. ○ Collect data anonymously to reduce the influence of social desirability bias. Collect data anonymously to reduce the influence of social desirability bias. Subjects are more likely to provide honest answers when they believe their responses cannot be traced back to them. Charboneau - AP Psychology - Manifest and Latent Content - James Wall Social Desirability Bias #shorts #research #researchmethodology The Social Desirability Response Set ○ a bias that occurs when individuals provide responses that they believe will be viewed favorably by others, rather than their true beliefs or opinions. ○ This bias can significantly distort survey results, as participants may be reluctant to admit negative or socially undesirable behaviors or attitudes. ○ Response styles can reduce the validity of responses if subjects are not responding to the manifest content, the plain meaning, of our questions in the way we intend. ○ Charboneau - AP Psychology - Manifest and Latent Content - James Wall ADVANTAGES DISADVANTAGES Anonymity Possibly lower response rate Convenience Respondents cannot ask for clarification. Limited depth Social Desirability Bias 2. Mail Surveys Considerations for Conducting Mail Surveys: 1. Be sure to include a polite and professional cover letter (and include a stamped, self-addressed envelope) 2. Anonymity 3. Expect lower response rates compared to other survey methods a. Consider incentives or follow-up mailings to improve rates. 4. Track unreturned questionnaire a. Second mailing 5. Keep mail survey short  ADVANTAGES DISADVANTAGES Anonymity Possibly lower response rate Convenience Limited control Accessibility Non-response bias ○ When a large number of individuals do not return their questionnaires, it can significantly affect the validity and generalizability of survey results. 3. Computer and Internet Surveys ADVANTAGES DISADVANTAGES Accessibility Limited Control Convenience ○ Duplicate responses Cheaper Access to internet Data Accuracy Technical issues Anonymity Inability to verify participant Reduced Coercion characteristics. 4. Telephone surveys are the most widely used method for conducting survey research, mainly assisted by computers in random digital dialing. Considerations For Telephone Surveys Interviewers should be able to establish rapport quickly with strangers and maintain a consistent interview style. Survey items and response formats need to be kept simple.  ADVANTAGES DISADVANTAGES High response rate Time constraints Highly accessible Ethical problems Provides anonymity to respondents 5. Interviews one of the best ways to gather high-quality survey data is to conduct face-to-face interviews. How to do a research interview TYPES OF INTERVIEWS ○ Structured interviews - the same questions are asked precisely the same way each time—deviations from this structure are not permitted. Provide more usable and quantifiable data. ○ Unstructured interviews - questions are more free-flowing and are not arranged in advance. The interviewer is free to explore interesting issues as they come up, but the information may not be usable for a content analysis. ADVANTAGES OF FACE-TO-FACE INTERVIEWS ADVANTAGES DISADVANTAGES Personal interaction Potential distractions Ability to build rapport Time and travel Pick-up of body language and non-verbal cue  ADVANTAGES OF ONLINE INTERVIEWS ADVANTAGES DISADVANTAGES Convenience Technical difficulties Increased access to remote Lack of personal candidates interaction 6. Focus groups are usually small groups of people with similar characteristics who are brought together by an interviewer (called a facilitator) who guides the group in a discussion of specific issues. What Are Focus Groups? ADVANTAGES DISADVANTAGES Less time-consuming Cannot ensure the anonymity of respondents Potential for observer dependency. 1. Reliability is the extent to which the survey is consistent and repeatable. Observe score = true score + error score ○ Observe score: the score that we measure or observe. It's the score a person gets on a test or assessment. ○ True score: the hypothetical score that reflects a person's true ability or knowledge ○ Error score: the difference between the observed score and the true score. It accounts for all the factors that can cause a person's observed score to deviate from their true ability  Types of reliability: 1. Internal Consistency Reliability ★ measures the consistency of items within a single instrument. 2. Test-Retest Reliability ★ Measures the consistency of scores over time. 3. Inter-Rater Reliability: ★ Measures the consistency of ratings or observations made by different individuals. 4. Validity refers to the extent to which a survey measures the intended topic Strategies for ensuring validity 1. Pretesting the questionnaire Does the survey measure what you want it to measure? Does performance on the survey predict actual behavior? Does it give the same results as other surveys designed to measure similar topics? Do the individual survey items fairly capture all the important aspects of the topic? 2. Adding bogus questions 3. Using standardized tests or surveys that have already been validated by others. Types of Validity: 1. Content Validity ★ assesses whether the survey items adequately represent the construct being measured. 2. Criterion Validity ★ evaluates the relationship between the survey scores and a criterion measure. 3. Construct Validity ★ evaluates whether the survey measures the intended psychological construct.  deciding who the subjects will be and selecting them Population - population consists of all people, animals, or objects that have at least one characteristic in common. Sample - is a group that is a subset of the population of interest. ○ Representativeness - how closely the sample mirrors the larger population. What is Sampling? PROBABILITY SAMPLING involves selecting subjects in such a way that the odds of their being in the study are known or can be calculated. Random selection ○ meaning that any member of the population has an equal opportunity to be selected. 1. Simple random sampling - a random selection from the entire population, where each unit has an equal chance of selection. 2. Systematic random sampling - a variation of random sampling in which a researcher selects every nth person from the population. 3. Stratified random sampling - is obtained by randomly sampling from people in each subgroup in the same proportions as they exist in the population. 4. Cluster sampling - a form of probability sampling in which a researcher samples entire clusters, or naturally occurring groups, that exist within the population. NONPROBABILITY SAMPLING A sampling process wherein probabilities of selection are not specified for the individual units in the population. When the researcher is not after generalizing the results of the study to the population or universe. 1. Quota Sampling The main concern in quota sampling is to come up with the desired number of samples no matter how they are selected. Example: 2. Convenience Sampling In this sampling method, the researcher selects respondents who are available at the time and place the data is to be collected. This is also called haphazard or incidental sampling. 3. Purposive Sampling The researcher selects those who can best help explain or give information based on his judgment. Researchers identify individuals who are experts or have unique experiences relevant to the study. This method of sampling is also called judgmental sampling. Example: ○ Exploring the Influence of Kpop Groups on Gen Z's Emotional Well-being.  4. Snowball Sampling This method is used where respondents are difficult to identify and best located through referral networks. Snowball sampling is used predominantely for sampling very small, uncommon, or unique populations at times when researchers do not know who the population members are or how to contact them. growing larger as it accumulates more snow. In the same way, the sampling process starts with a small group of individuals, and then these individuals introduce others to the study, causing the sample to grow larger and larger. Example: ○ Delving into the Minds of Individuals Involved in Illegal Activities. The way a sample is chosen influences what can be concluded from the results. The researchers must explain: ○ How the sample was selected, including the population sampled ○ recruitment methods ○ demographic characteristics of the participants. ○ Any incentives or requirements for participation.  REVIEW QUESTIONS 1. What is the primary purpose of survey research? a. To conduct experiments. b. To observe behavior directly and formulate a hypothesis. c. To gather information about people's opinions, attitudes, and behaviors. d. To test hypotheses about causal relationships. 2. Which type of scale has equal intervals between values and a true zero point? a. Nominal b. Ratio c. Ordinal d. Interval 3. When choosing a level of measurement, what is the general rule to follow? a. Always choose the lowest level possible. b. Choose the level that is most convenient for data analysis. c. Choose the level that provides the most information about the response. d. Choose the level that you want. 4. You are conducting a survey to measure people's attitudes towards abortion. Which survey method would be most effective in minimizing social desirability bias and ensuring high response rates, while also considering the potential for reactivity? a. Internet survey with anonymity guaranteed b. Mail survey with a monetary incentive c. Face-to-face interview in a group setting d. Telephone survey with a live interviewer 5. You are conducting a survey to measure the effectiveness of a new reading intervention program. You randomly select 100 students from the University of Makati HSU. Which of the following details about your sampling method would be most important to report in your research report? a. The number of students who completed the entire program b. The average age of the students who participated c. The number of students who got higher grades in English. d. The type of sampling method used (e.g., simple random sampling, stratified sampling) 6. In experimental research, a group of psychology students would like to collect respondents in a randomized controlled method by computing for the nth person or k interval, ensuring that their respondents would evenly represent their targeted population. What type of sampling technique should the students use? a. Convenience sampling b. Systematic random sampling  c. Simple random sampling d. Purposive sampling 7. Researchers are responsible for properly constructing their survey questions that align with their research objectives. In the case of 2-C psychology students constructing their survey items, what do they have to consider if their research topic is about the feedback of University of Makati students on food cafeteria? a. Open-ended questions b. Nominal questions c. Assorted questions d. Both A and B 8. Psychology students are done determining their target population and are now proceeding to identify sample respondents. What do they have to consider in choosing individuals for their sample? a. Representativeness b. Appearance c. Context effects d. Both A and C 9. When constructing a survey in the form of a close-ended question, a researcher must be able to exhaust all possible options there is. To do so, researchers should be aware of the respondents’ _________. a. Value-laden b. Response style(s) c. Personality d. All of the above 10. A group of researchers is conducting their research on the perceptions of students on makeup usage policies at the University of Makati and are utilizing an interview as their collection method. Throughout the interview, the researchers (moderator) didn’t follow a script and kept the questions free-flowing. What type of interview style did the researcher use? a. Free-flowing style b. Unstructured interview c. Structured interview d. Contextual style KEY ANSWERS 1. C 2. B 3. C 4. A 5. D 6. B 7. C 8. A 9. B 10. B