Marketing Research: Survey Methods, Measurement, and Questionnaires PDF

Marketing Research Lecture 4: Survey Methods, Measurement, and Questionnaires Professor Landry  Reminders/Announcements Next week: online class for pilot surveys zoom link on quercus In two weeks: midterm during class  Group Meetings (with me) 15-20 minute m...

Marketing Research Lecture 4: Survey Methods, Measurement, and Questionnaires Professor Landry  Reminders/Announcements Next week: online class for pilot surveys zoom link on quercus In two weeks: midterm during class  Group Meetings (with me) 15-20 minute meetings to get feedback on your proposal Available dates: Feb 19 (Wed), 2pm-7pm Feb 20 (Thurs), 12pm-5pm Feb 26 (Wed) during class hours and 1pm-3pm Feb 27 (Thurs), 12pm-5pm Email me as a group with 3 or more hour-long time slots that would work (first-come, first-serve) Location: this classroom if during class hours, otherwise KN 245  Feb 26: no lecture you only need to attend for your scheduled meeting (if any) Suppose you pick up your mail and find a survey that would take 5 minutes to complete. Would you complete the survey? Less More Topic Likely Likely TikTok parking prices at UTM life insurance  Survey Methods Survey: interview many respondents using a predesigned questionnaire Person-administered: an interviewer reads questions, either face-to-face or over the phone, to the respondent and records his or her answers  Person-Administered Surveys: Pros and Cons Advantages: Disadvantages: Feedback/adaptability Can be prone to human error Rapport Control of “interviewer effects” – responses may depend on interviewer’s Quality control personality or how questions were asked Relatively good response rates Relatively costly “Interview evaluation”: subjects may be inclined to give answer perceived as “correct” to interviewer. Especially a problem with sensitive topics  Survey Methods Survey: interview many respondents using a predesigned questionnaire Person-administered: an interviewer reads questions, either face-to-face or over the phone, to the respondent and records his or her answers Computer-administered: computer technology used to pose questions and record responses Popular online survey sites: Qualtrics, Survey Monkey, Google Forms, MS Forms… Computer-Administered Surveys: Pros & Cons Advantages: Disadvantages: Breadth of user-friendly features Requires computer-literate and/or internet-connected Relatively inexpensive respondents Reduction of interview Cooperation can be an issue evaluation effects Automatic data input/collection  Survey Methods Survey: interview many respondents using a predesigned questionnaire Person administered: an interviewer reads questions, either face-to-face or over the phone, to the respondent and records his or her answers Computer administered: computer technology used to pose questions and record responses Popular online survey sites: Qualtrics, Survey Monkey, Google Forms, MS Forms… Self-administered: respondents complete own survey Self-Administered Surveys: Pros & Cons Advantages: Disadvantages: Reduced cost Nonresponse can exacerbate self-selection bias Respondents control pace at which they answer Lack of monitoring—no one to explain or answer questions No interview-evaluation apprehension High questionnaire requirements – it must be “perfect” “Nonresponse can exacerbate self-selection bias” Are there systematic If the answer is yes, differences between the sample may not those who respond to be representative of the survey and those the population that who do not respond? you intend to study Suppose you pick up your mail and find a survey that would take 5 minutes to complete. Would you complete the survey? Less More Topic Likely Likely TikTok parking prices at UTM life insurance  Which group/letter best describes you? I’m _________ to do a 5-min TikTok survey less likely more likely Overall, I have positive A B a _______ view of TikTok negative C D “Nonresponse can exacerbate self-selection bias” Are there systematic If the answer is yes, differences between the sample may not those who respond to be representative of the survey and those the population that who do not respond? you intend to study Would self-selection in this case be a problem? Why or why not?  Which group/letter best describes you? I’m _________ to do a 5-min survey about UTM parking prices less likely more likely Overall, I have _______ opinions strong A B about the parking prices at UTM mild C D “Nonresponse can exacerbate self-selection bias” Are there systematic If the answer is yes, differences between the sample may not those who respond to be representative of the survey and those the population that who do not respond? you intend to study Would self-selection in this case be a problem? Why or why not? Suppose you pick up your mail and find a survey that would take 5 minutes to complete. Would you complete the survey? Less More Topic Likely Likely TikTok parking prices at UTM life insurance What about life insurance?  Survey Methods Survey: interview many respondents using a predesigned questionnaire Person administered: an interviewer reads questions, either face-to-face or over the phone, to the respondent and records his or her answers Computer administered: computer technology used to pose questions and record responses Popular online survey sites: Qualtrics, Survey Monkey, Google Forms, MS Forms… Self-administered: respondents complete own survey Mixed mode: a combination of methods  Mixed-Mode Surveys: Pros & Cons Advantages: Multiple advantages to achieve data collection goal Disadvantages: E.g. may use online surveys to “Mode” may affect response quickly reach portion of Additional complexity population and telephone to reach those unreachable via online surveys Can alleviate problems associated with any one method  Mode May Affect Response? Pew Study: phone surveys and online surveys can yield different results One notable difference: more “negativity” in online vs phone survey responses Those surveyed via phone (vs. online) are substantially more likely to say they are “very satisfied” with their family and social life, and substantially less likely to express “very unfavorable” opinions of politicians  Mode May Affect Response? Why would responses be different between an online survey and a phone survey? Should we believe the phone responses or the online responses? Is “mode may affect response” only a disadvantage for mixed-mode surveys? Is it possible that respondents are answering the same way on the phone that they would have if they were asked online, and vice versa? Measurement: determining whether or how much of a property is possessed by an object Objective properties are physically verifiable such as age, height, or number of bottles purchased Subjective properties: mental constructs that are not directly observable such as attitudes, opinions, or intentions translated onto an “intensity continuum” Scale development: designing questions and response formats to measure subjective properties Types of Measures: 1. Nominal: use only labels 2. Ordinal: responses can be ranked 3. Scale: distance between levels is known… Ratio: a true zero exists Interval: equal distances between levels (implicit)  Measures: Examples Nominal How many years old are you? Ordinal Are you left-handed, right-handed, or ambidextrous? Interval Rank the following from most- to least- favorite: apples, oranges, pears Ratio On a 1-10 scale with 1 being the worst and 10 being the best, how would you rate our customer service?  Smartphone brand preferences: which type of measure would you use? Nominal Which phone brand do you like most? Choose one. Rank these four phone brands from 1 to 4, with 1 Ordinal being most preferred and 4 being least preferred. Indicate how much you like each brand from 1 to 5, Interval 1 being “don’t like it at all”, 5 being “like it a lot”. Ratio Indicate how likely (in percent) you would be to choose each brand if you were to buy a new phone Likert scale: respondents indicate degree of (dis)agreement on a symmetric agree–disagree scale for each of a series of statements Common application: lifestyle inventories Used for psychographic segmentation, i.e. segmenting the market based on personality traits, interests, values, and lifestyles Features statements like “I enjoy going to the mall,” “Environmental issues are important to me,” or “I consider myself to be a proactive individual” Semantic differential scales feature a sequence of bipolar Halo effect: a general adjectives for an objects’ overall like or dislike properties and respondents of the object could indicate their impressions of bias responses on its each property on a continuum specific properties  Why does this happen? …one of the most polarizing brands – many people love McDonalds, but Why is it a problem? just as many hate McDonalds (HBR) What can we do? Good taste ⃝ ⃝ ⃝ ⃝ ⃝ Bad taste Friendly workers ⃝ ⃝ ⃝ ⃝ ⃝ Unfriendly workers Healthy food ⃝ ⃝ ⃝ ⃝ ⃝ Unhealthy food Pleasant ambience ⃝ ⃝ ⃝ ⃝ ⃝ Unpleasant ambience High-quality ingredients ⃝ ⃝ ⃝ ⃝ ⃝ Low-quality ingredients Fast service ⃝ ⃝ ⃝ ⃝ ⃝ Slow service Refreshing Soft Drinks ⃝ ⃝ ⃝ ⃝ ⃝ Unrefreshing Soft Drinks  a b What can we do? 1a 1b Good taste ⃝ ⃝ ⃝ ⃝ ⃝ Bad taste 2a 2b Friendly workers ⃝ ⃝ ⃝ ⃝ ⃝ Unfriendly workers 3a 3b Unhealthy food ⃝ ⃝ ⃝ ⃝ ⃝ Healthy food 4a 4b Pleasant ambience ⃝ ⃝ ⃝ ⃝ ⃝ Unpleasant ambience 5a 5b Low-quality ingredients ⃝ ⃝ ⃝ ⃝ ⃝ High-quality ingredients 6a 6b Fast service ⃝ ⃝ ⃝ ⃝ ⃝ Slow service 7a 7b Unrefreshing Soft Drinks ⃝ ⃝ ⃝ ⃝ ⃝ Refreshing Soft Drinks Questionnaire design process by which the researcher determines question format, wording, and organizes the layout of questionnaire Questionnaire organization refers to the sequence of questions Poorly organized questionnaires discourage and frustrate respondents and may even cause them to stop answering in the middle of the survey Questionnaire development selecting response formats while wording questions to be understandable, unambiguous, and unbiased The researcher should strive to minimize question bias, i.e. the ability of a question’s wording or format to influence respondents’ answers  Types of Questions Warm-up questions: simple, easy-to-answer questions used to get respondents interested and to demonstrate the ease of continuing and completing the survey Can also provide information relevant to the research objective Examples: “what social media platforms do you use?”, “how often do you use social media?”, “what is your favorite social media platform?” For person-administered surveys, in which a respondent would typically verbalize (rather than type or write) their answers, warm-up questions are often open-ended.  Types of Questions Classification/demographic questions: used to classify respondents into various groups for purposes of analysis Examples: questions about age, gender, income level, etc. Often “personal”  Types of Questions Screening questions: questions used to “screen out” respondents who are not eligible to participate in the study “Eligibility” defined in advance by the researcher depending on the research objectives For example, a survey intended for dog owners could ask “Do you own a dog?” as a screening question; those who answer “no” would not continue with the survey  Types of Questions Transitions: statements or questions used to inform the respondent of upcoming changes in topic or format Examples: “Now that we’ve discussed your general shopping habits, not let’s focus specifically on your experiences with Amazon”; “Next, you are going to be shown statements and, for each statement, you will be asked to indicate whether you agree or disagree with that statement.”  Types of Questions “Tough” questions: questions that are more complicated or difficult-to- answer, but are very relevant to the researcher and helpful in addressing the objectives of the research Should only use a tough question when it provides crucial information that could not be obtained with an easier question Example: “What would be your maximum willingness to pay, in dollars, for a subscription to a MGT 453 fruit service that provided bananas during class breaks?”  Ordering of Questions first question(s) warm-up right after any questions of first type classification/ demographic start of a new section featuring a new topic or question format screening middle or towards end of survey transition last question(s) tough Questionnaire design process by which the researcher determines question format, wording, and organizes the layout of questionnaire Questionnaire organization refers to the sequence of questions Poorly organized questionnaires discourage and frustrate respondents and may even cause them to stop answering in the middle of the survey Questionnaire development selecting response formats while wording questions to be understandable, unambiguous, and unbiased The researcher should strive to minimize question bias, i.e. the ability of a question’s wording or format to influence respondents’ answers  Example Questions How many times have you been to Tim Hortons in the last month? a) Never b) 1-4 times c) 5-9 times d) 10-20 times  Example Questions How old are you? a) 0-10 b) 10-20 c) 20-30 d) 30-40 e) 40+  Example Questions With the flu season approaching quickly, do you plan to get a flu shot in the next two weeks? a) Yes b) No  Example Questions In the last five days, how many minutes did you spend watching television? _________  Example Questions Was the customer service agent you spoke with today able to resolve your problem in a professional manner? a) Yes b) No  Example Questions Was your limousine chauffer punctual? a) Yes b) No  Example Questions Fill in the circle that describes your experience at Canadian Tire today Very Disappointed Disappointed Neutral Satisfied Very Satisfied ⃝ ⃝ ⃝ ⃝ ⃝  Example Question Do you floss at least once a week? a) Yes b) No Veiled Reports for Sensitive Questions (Coffman et al., 2015) Direct Report I own an electric toothbrush I like cinnamon-flavored toothpaste I have never had wisdom teeth removed I believe the public supply of drinking water should be fluoridated How many of the statements above apply to you? (Please circle your answer) 0 1 2 3 4 Do you floss at least once a week? Yes No Veiled Reports for Sensitive Questions (Coffman et al., 2015) Veiled Report I own an electric toothbrush I like cinnamon-flavored toothpaste I have never had wisdom teeth removed I believe the public supply of drinking water should be fluoridated I floss at least once a week How many of the statements above apply to you? (Please circle your answer) 0 1 2 3 4 5  Veiled Reports for Sensitive Questions Let 𝑁𝐷 be the average response to the question “how many of the statements above apply to you?” on the direct report Let 𝑁𝑉 be the average response to the question “how many of the statements above apply to you?” on the veiled report Estimated % who floss at least once a week: 𝑁𝑉 − 𝑁𝐷