L6.pdf

Lecture 6: Data Collection II – Survey Design PROFESSOR JIA LIU  From Last Class § What are different types of Problem formulation measurement scales? Research design § What are the scales to measure unobserved...

Lecture 6: Data Collection II – Survey Design PROFESSOR JIA LIU  From Last Class § What are different types of Problem formulation measurement scales? Research design § What are the scales to measure unobserved Data collection concepts? Analysis and interpretation Reporting results HKUST, MARKETING RESEARCH  Objectives of a Questionnaire § Translate research objective to specific questions § Primary goals 1. Collect needed data 2. Minimize error § Other benefits/goals 1. Serve as permanent record 2. Facilitate data analysis 3. Collect information for reliability measure HKUST, MARKETING RESEARCH  Questionnaire Design 1. Specify the information needed 2. Determine survey method 3. Question content 4. Question structure/sequence 5. Question layout and recruiting message 6. Pre-testing HKUST, MARKETING RESEARCH  1. SPECIFY THE INFORMATION NEED HKUST, MARKETING RESEARCH  Specify the Information Need § Use the following input 1. Research problems 2. Use input from exploratory research § Summarize in conceptual model 1. Figure that links independent (X) and dependent variables (Y) 2. It summarizes your hypotheses HKUST, MARKETING RESEARCH  Conceptual Model Environmental consciousness H2 + Timing of the H1 - Intention to event participate Received H3 + promotional email HKUST, MARKETING RESEARCH  2. DETERMINE SURVEY METHOD HKUST, MARKETING RESEARCH  Determine Survey Method Personal Mail Telephone Internet In-Home PAPI Traditional Email CAPI Mail Panel CATI Web Survey Web Panel HKUST, MARKETING RESEARCH Criteria for Evaluating Survey Method § Cost § Perceived anonymity of the § Time respondent § Response rate § Potential for interviewer bias § Sample control § Quantity of data § Diversity of questions § Use of physical stimuli HKUST, MARKETING RESEARCH  Ways to Improve Data Quality § Incentive-aligned survey design § Filler questions § Attention check questions (remove subjects who failed) § Specific instruction on how to answer a question with choices and also an open-ended box § e.g., please type XXX in the box § Recall question § e.g., ask about the ad or stimulus they saw before HKUST, MARKETING RESEARCH  3. DETERMINE QUESTION CONTENT HKUST, MARKETING RESEARCH  Question Content The way a question is asked can dramatically change the conclusion we make Wording effects 1. “Do you think the United States will go into the war before it is over?” Yes (41%) No (33%) Don’t know (26%) 2. “Do you think the united states will succeed in staying out of the war?” Yes (44%) No (30%) Don’t know (26%) HKUST, MARKETING RESEARCH HKUST, MARKETING RESEARCH  What Do You Think of the Following Questions? 1. Do you go to the movies frequently? 2. Do you think Coca Cola is tasty and refreshing? 3. Don’t you think that investment in children’s future is an important goal for the government? 4. Which brand of shampoo do you use? HKUST, MARKETING RESEARCH  What Do You Think of the Following Questions? 5. What is the annual spending per person on groceries in your household? 6. Did you cheat during the quiz? 7. Why did you buy a Samsung HD TV? q Trendy design q Feel it represents the highest quality q Picture quality is great q Very durable q Other reason HKUST, MARKETING RESEARCH  More Suggestions for Question Wording 1. If possible, use interval scales (Likert or semantic differential scales) or ratio scales 2. Limit the length and time frame of the questions Rules of thumb: < 15 words < one full line in print 3. Be very specific about the product/service when asking about purchase likelihood 4. Ask about others for sensitive questions HKUST, MARKETING RESEARCH  4. DETERMINE QUESTION SEQUENCE HKUST, MARKETING RESEARCH  Question Sequence Location Type Rationale Qualifying Screeners To identify target respondents questions Easy to answer. Shows respondents First few Warm-ups survey is simple Relate to research objectives, more First third Focus Qs effort needed to answer Difficult and Has committed to completing survey; Middle complicated Can see just a few question are left Demographic Respondents may leave these Last section and sensitive questions blank HKUST, MARKETING RESEARCH  Order Bias Experiment (Strack, Martin and Schwarz 1988, European Journal of Social Psychology): 180 students were approached and asked to answer a questionnaire with 15 questions. Each student received randomly one of two questionnaires. The only difference between the questionnaires was that question 7 and 8 were reversed. Condition 1 (control group): Condition 2 (treatment group): HKUST, MARKETING RESEARCH  5. QUESTION LAYOUT AND RECRUITING MESSAGE HKUST, MARKETING RESEARCH  Layout and Recruiting Message Physical appearance can be important for participation and completion May include Introduction/cover letter Purpose of the survey How respondents were selected Request for participation Confidentiality Thanking the respondent HKUST, MARKETING RESEARCH  Cover Letter HKUST, MARKETING RESEARCH  6. PRE-TESTING HKUST, MARKETING RESEARCH  Pre-Testing Test the questionnaire on a small sample Sample members should be actual members of the target population Ensures the questions you intend are being answered Screens for problems with flow patterns Pay attention to: How long did it take? (