Questionnaire Design PDF

Describing the Questionnaire Developing the Questionnaire 1. Planning what to measure 2. Formulating questions 3. Decide on, Order wording Layout 4. Pretesting/Evaluation Cycle, fix whatever we find Step 1: Planning what to measure Determine survey objectives, resources and constra...

Describing the Questionnaire Developing the Questionnaire 1. Planning what to measure 2. Formulating questions 3. Decide on, Order wording Layout 4. Pretesting/Evaluation Cycle, fix whatever we find Step 1: Planning what to measure Determine survey objectives, resources and constraints - Alignment with research purpose and objectives - Have a clear understanding of what information is needed, recall what the questions that the reach project is looking to answer - Alignment with data collection method Step 2: Formulating Questions Degree of freedom to respondents (based on research approach) - Open ended questions?, when given exploratory approach - Close ended questions? - A combination of both open and close ended? Open ended Questions (step 2) - Maximum degree of freedom - Respondents can answer in their own words, more confidence - Minimize researcher induced bias - They are exploratory in nature. Help to anticipate a good range of responses - Advantages ➔ Respondents feel free to answer (no constraints or limitation) ➔ Potential to vet deeper insights or additional alternatives ➔ Allows for probing (only for personal interview) - Disadvantages ➔ Answers are more difficult to interpret ➔ Editing and coding can be challenging and subjective ➔ Potential for interview bias ➔ Answer may be shallow or insufficient ➔ Lower response rate Ex. how was you shopping experience today - It was a hassle - Slight better than last week Closed-ended questions advantages (step 2) - Reduced degree of freedom - Respondents forced to provide an answer based on a limited number of choices - 2 main ways ➔ Choice from a list (nominal scale) ➔ Single-choice rating on a scale (interval scale) - Advantages: ➔ easy, accurate data coding and entry ➔ Limited number of responses ➔ List of options may help with respondent recall ➔ Reduced interviewer bias - Disadvantages ➔ Planning stage has to be extensive ➔ Higher burden on research at the planning stage (remember validity, reliability, sensitivity, generalizability and relevancy?) ➔ Choices provided by the research may be no exhaustive ➔ Respondents must select from given alternatives ➔ No freedom for respondent while answering - Type of Closed ended questions ➔ Choice: Dichotomous choice questions: ❖ Simplest form of closed ended question ❖ Respondents are given only TWO choices to select form ❖ Typically used to collect demographic information, perform screening (sample validation) ❖ Typically coded using 0 and 1 or -1 and 1 in order to perform data analysis Multiple choice questions ❖ Allow respondents to choose from several (more than 2) alternatives ❖ Typically used to collect demographic, psychographic and preference information ❖ Typically, categories are coded from 1- n …. Where n= number of categories in the set ➔ Rank: ranking questions Determine the relative importance to respondents of various options They serve to compare preference in choices  ➔ Rate: scaled-response questions Questions format the permits the measurement of the level or degree of “intensity” of a respondent’s answers Step 3: Wording your questions The way you phrase the syntax of your questions may impact your results Avoid 1. Leading questions 2. High complexity 3. Making assumptions 4. Ambiguity 5. Burdensome questions 6. Double barreled items Leading Questions - You enjoyed our services didn’t you? ➔ Did you enjoy our services (agreement scale) (good) - How exciting has this event been for you? ➔ Did you find this even exciting (agreement scale) (good) High Complexity Questions - How likely would you be to buy an iphone docking station with 11 x 6.5 dimensions that utilizes wired technology for the dock and infrared technology for the remote (bad) ➔ How likely would be be to buy and iphone docking station that is the size of a textbook, has wired docking cradle and a remote (good) Making assumptions - Should Macy’s continue its excellent gift-wrapping program? Yes or no ➔ In the past year, how many times did you use Macy’s gift-wrapping program? Should Macy’s continue this program Yes or no Ambiguous questions  - How many times per month do you visit fast-food restaurants? Never/ Occasionally/ Sometimes/ Often ➔ How many times per month do you visit a fast food restaurant Less than once/ 1 or 2 times/ 3 or 4 times/ more than 4 times Burdensome Questions - Do you recall any commercials on the program? What were the brands advertised? ➔ Do you recall whether there was a brand of beer advertised on that program? ➔ I will read you a list of beer names. Can you pick out the name of the beer that was advertised on the program Double barrel Questions - Are you satisfied with the price and the service of Taco Bell? (they are being asked the price and the service) ➔ Please rate your level of satisfaction with each item below regarding taco bell Step 3: Questionnaire design Concerns  Step 3 (sensitive questions) ○ Directly asking questions like the ones below will yield meaningless responses - Have you ever defaulted on a credit account - Do you smoke pot? Long, open-ended questions with familiar wording have been found effective when asking sensitive questions Examples - The casual approach: “Have you eaten “Frosted Flakes” within the last week?  - Everybody approaches: “As you know, some people have been eating frosted flakes for breakfast. Do you eat frosted flakes? - The “other people” approach: “do you know any adults who eat Frosted Flakes?” “How about yourself?” - The sealed ballot approach: explain that the survey respects people’s right to anonymity with respect to their eating habits and respondents themselves are to ➔ Fill out the answer to the question ➔ Seal it in an envelope ➔ And drop it in a box conspicuously labeled “sealed ballot box” carried by the interviewer Typical Questionnaire Layout (step 3) 1. Cover letter/introduction 2. Screening (to evaluate if the person is part of your target population) 2.1 “Bot Screening” (online) - “I'm not a bot” 3. “Warm up” Questions - “What brand of X do you own?” 4. “Involving” questions - “Please rate X on the following” 5. Insert Prompters - “Now, we would like to ask…” 6. “Sensitive” Questions at the end - “Please rate X on the following” 7. Demographics 8. Closing remarks/thank you Step 4: Pretest and Evaluation Cycle