Survey: Questionnaire Design PDF

Survey: Questionnaire Design MKTG 351: MARKETING RESEARCH CONDUCT: Prepare Materials OBJECTIVES OF QUESTIONNAIRE DESIGN 1. Set of questions design to generate the data necessary to accomplish the objectives of the research project 2. Standardizes the wording and sequence o...

Survey: Questionnaire Design MKTG 351: MARKETING RESEARCH CONDUCT: Prepare Materials OBJECTIVES OF QUESTIONNAIRE DESIGN 1. Set of questions design to generate the data necessary to accomplish the objectives of the research project 2. Standardizes the wording and sequence of the questions 3. Easy to code and translate into managerial recommendations Basic Considerations: Questionnaire Design What should be asked? How should the questions be phrased? In what sequence should the questions be arranged? Does the questionnaire need to be revised? How should the questionnaire be pretested? What questionnaire layout will be the best? 3 CONSIDERATION 1. What should be asked? The focus here is on: RELEVANCY Collects all information necessary to: ◦ Address the Research Questions ◦ Describe the Sample CONSIDERATION 2. How should questions be phrased? Avoid Problem Questions: 1. Complexity Phrasing includes: 2. Leading ◦ Words used to write the question 3. Loaded ◦ Scale selected to measure their 4. Ambiguity answer 5. Double-barreled 6. Assumptions 7. Burdensome Focus on being: ◦ Clear – in terms of being understood Scale Choices: ◦ Unbiased – in question and answer 1. Select choices 2. Ranking 3. Rating ◦ Answerable – ability and willingness Category Likert Semantic Differential REVIEW: problematic questions AVOID COMPLEXITY! AVOID LEADING QUESTIONS! Do you think that the soft Experts believe that all drink distribution is consumers should comparison adequate? shop. Do you agree? Problematic because: Problematic because: It suggests or implies certain answers. No one understands what you mean. Possible Solution: Possible Solution: All consumers should comparison shop. Are soft drinks easy to find whenever you want to buy one? Not all consumers should comparison shop. REVIEW: problematic questions AVOID AMBIGUITY! AVOID DOUBLE-BARRELED ITEMS! Does your company provide training Do you use drugs? for new employees and retraining for existing staff? Problematic because: It is unclear Problematic because: as to what is being asked. It covers two issues at the same time. Possible Solution: Possible Solution: Does your company provide training programs for Do you use prescription drugs? new employees? Do you use over the counter drugs? Does your company provide training programs for existing staff? Do you use illegal drugs? REVIEW: problematic questions AVOID MAKING ASSUMPTIONS! AVOID BURDENSOME QUESTIONS! Is our new website an How many sporting events improvement over our last did you watch in the past website? year? Problematic because: It takes for granted a pre-existing Problematic because: familiarity, knowledge, or opinion on a No one has that great of a memory. topic. Possible Solution: Possible Solution: How regularly do you watch NCAA Did you use our last website? basketball games? REVIEW: problematic questions WHAT ABOUT SENSITIVE AVOID LOADED QUESTIONS! QUESTIONS? For these types of questions, respondents feel uncomfortable providing an answer. How important is Why? education? ◦ Topic is emotionally charged. ◦ There is a social desirable answer. ◦ There is a concern over privacy. (It’s none of your business.) Problematic because: It suggests a socially desirable How to deal? answer or is emotionally charged. ◦ Avoid topics (not always possible) ◦ Promise (and provide) anonymity and confidentiality Possible Solution: ◦ Be conscientious of placement within the survey (Not first; How important is education Not last… build trust before asking) compared to other major social ◦ Use phrasing to allow respondent to be comfortable issues? with any of the answer choices  Task where the respondent must choice REVIEW: scale between two or more alternatives Assumes that whatever is chosen is the choices CHOICE scales: preferred option of the respondent Yes/No is a Choice scale PICK ONE PICK ALL THAT APPLY  Task where respondents puts a small REVIEW: scale number of objects in order based on preference or some other choices RANKING scales: characteristic. Ranking results are only relevant in the context of the choices provided. RANK ORDER CONSTANT-SUM  the magnitude of a characteristic of an REVIEW: scale object. Provides flexibility in format and analysis choices RATING scales: Tends to be most commonly used scales 5 point scales (Greater sensitivity: Use 7 point) SEMANTIC CATEGORY LIKERT DIFFERENTIAL Statements (items) are Statements (items) are Statements (items) evaluated on a evaluated on a continuum expressing favor or continuum bounded by polar disfavor are evaluated adjectives based on level of agreement Examples: Examples: ◦ Satisfaction ◦ Positive – Negative ◦ Importance ◦ Strong – Weak ◦ Frequency ◦ Active – Passive ◦ Likelihood ◦ Dirty - Clean CONSIDERATION 3. What sequence should the questions be arranged in? Order Bias The order that you ask your questions or provide your answers affect, unintentionally or intentionally, the responses that a respondent provides ◦ Occurs primarily when: 1. Prior questions affect the answers to later questions 2. Order of answer choices (e.g., first, second, third, etc.) influences which answer is selected ◦ Example: 1. Please rate the temperature of your entrée. 2. Please rate the taste of your food. 3. Please rate the menu selection here. 4. Please rate the courtesy of your server. 5. Please rate the service you received. 6. Please rate your overall experience at this restaurant Sequence: Some Guidelines Filter questions: A question that screens out respondents who are not qualified to participate in a survey or to answer a question on a survey. General Guidelines: 1. Group survey questions by topic area 2. Funnel Approach (Technique): Organize from broad to specific 3. Flow smoothly and logically – use transitions! Specific Guidelines: 1. Topic focused questions to start (basic information) 2. If asking sensitive or difficult questions, place them near the end 3. Demographic questions at the end (classification information) CONSIDERATION 4. Does the questionnaire need to be revised? There will – and should – be several What should be asked? rounds of revisions for your questionnaire. How should the Exact number depends on the questions be judgments of researchers and clients. phrased? Revision process ends when In what agreement is reached. sequence should the questions be arranged? Does the questionnaire need to be Yes! revised? CONSIDERATION 5. How should the questionnaire be pretested? Pretest ◦ Seeks to determine whether respondents have any difficulty understanding the questionnaire and whether there are any missing, ambiguous, biased, or just bad questions on the survey. How: ◦ Experts ◦ “Informed” Respondents ◦ Respondents Pretest: What to do FOCUS ON SPECIFIC QUESTIONS: FOCUS ON OVERALL SURVEY: Key Questions: Key Questions: ◦ Do they get what I am asking? ◦ Does my survey work? ◦ Am I asking all that I need to? ◦ Do I have any mistakes? What to look for: What to look for? ◦ Understanding ◦ Interest and Attention ◦ Task Difficulty ◦ Length ◦ Variation in Responses ◦ Flow ◦ Add/Delete Questions ◦ Skip Patterns CONSIDERATION 6. What questionnaire layout will be the best? Focus on “Readability”: ◦ Quality of the paper/website ◦ Clarity of reproduction ◦ Crowding of questions/answer choices General Guidelines: ◦ Allow plenty of space ◦ Easy to read font and color contrast ◦ Use “extras” sparingly SUMMARY: The Basic Considerations of QD RELEVANCY! What should be asked? Collects all information necessary to address a research question and describe the sample Includes both the Words to write the Question and the Scale to measure their How should the Answer questions be phrased? Focus on: (1) Clarity; (2) Unbiased; (3) Answerable In what sequence should Utilize Funnel Technique (broad to specific) and group by topic area. the questions be Demographics at the end. Main concern is Order Bias arranged? Yes. Always. Multiple Times. Does the questionnaire Exact number depends on researcher and client. Ends when agreement is need to be revised? reached. How should the Determine whether there are any missing, ambiguous, biased, or just bad questionnaire be questions on the survey. Use: (1) Experts; (2) “Informed” Respondents; (3) Respondents pretested? What questionnaire Depends on Selected Method: Interviewer-Administered vs. Self-Administered Focus on: (1) Clarity; (2) Usability; (3) ‘Complete’-ability layout will be the best? 19