ACM 3 - Interviews and Surveys - Copy (1) PDF

4H1ACM Analyzing & Understanding Markets Analyzing and Understanding Markets: Interviews and Surveys What we did last time ▪ Market research can be carried out internally, or externally through specialized service providers. ▪ Market research is a process, comprising several steps from problem...

4H1ACM Analyzing & Understanding Markets Analyzing and Understanding Markets: Interviews and Surveys What we did last time ▪ Market research can be carried out internally, or externally through specialized service providers. ▪ Market research is a process, comprising several steps from problem formulation and research design to the data gathering, analysis and reporting. ▪ Research designs can be based on secondary or primary data; they can be exploratory, descriptive, or causal. ▪ Last time, we focused on the use of secondary data (desk research). We examined different types of data sources, and how to best use the emlyon library resources to find good data for your first assignment. We’ll now turn to examine how to gather primary data. Agenda ▪ Qualitative Studies ▪ Surveys ▪ Characteristics ▪ Sampling ▪ How to administer surveys ▪ Typologies of questions ▪ Creating a questionnaire ▪ Group Assignment 2: Guidelines ▪ Instructions for next time (student presentations) ▪ Time for groupwork Qualitative studies Qualitative studies The goal of qualitative studies is to understand a little-understood phenomenon, such as: ▪ the decision process to purchase an electric car ▪ why consumers are not satisfied with a product or service ▪ the reasons why consumers switch from Apple to Android (or viceversa) Qualitative studies do not quantify such phenomena – but rather focus on understand their qualities ▪ For example, if the goal is to understand why consumers are not satisfied, your key findings will be to identify the majour sources of dissatisfaction – for example, 1, the product is too complex to use; 2, maintenance and spare parts are expensive; 3, there are too many plastic components so it’s not eco-friendly. ▪ If your goal is to understand the consumer decision making process, you’ll identify what are the benefits sought and the criteria used to compare product alternatives. What is the relationship between qual and quant studies? Qualitative studies are sometimes sufficient to generate insight on little understood issues – but they can be carried out together with quantitative studies. Before quantitative studies, qual studies permit to: ▪ Better understand the issues at stake ▪ Generate answers to multiple-choice questions in surveys After quantitative studies, qualitative studies permit to: ▪ Better understand quantitative research findings ▪ Better understand answering patterns ▪ Illustrate survey answers with key sentences (verbatims) from consumers. Main typologies of qualitative studies Ethnographic and observation-based studies ▪ They involve observing consumers in their natural habitat – for example their home or points of sale or when they attend events. ▪ They allow to observe first-hand how people consume products, services, experiences in the natural environment when consumption takes place – rather than relying on people’s explanation of their behaviors. ▪ Based on anthropological methods; nowadays used by large corporations. Individual, in-depth interviews ▪ They involve asking participants (also called informants) a series a questions. ▪ Can be more or less structured, depending on how much is know about the issue at stake. ▪ Increasingly, they are carried out online (e.g., zoom). Focus groups and other forms of group interviews ▪ Small group of participants ▪ Animated by a moderator ▪ Interaction among participants allows to obtain information that wouldn’t otherwise emerge. Individual in-depth interviews They are semi-structured. ▪ The interview guide consists in a sequence of open-ended questions, i.e., with no choice of answers, and formulated in such a way as to encourage the person to expand on the subject ▪ Sample is often around 20 people ▪ The sample should not be representative of the population, but rather of the phenomenon under study (e.g., if the focus is on the decision process of buying electric cars, we will only include people who have recently bought one). ▪ Interviews are generally face-to-face or with video-conferencing tools – rarely by telephone ▪ Questions can be sent in advance, if some preparation is needed ▪ Interviews are recorded (audio and/or video) ▪ Interviews are transcribed ▪ There are voice recognition software programs (for a fee, e.g. Happy Scribe) ▪ Interviews are transcribed verbatim (from Latin, word by word). ▪ Analysis of transcripts ▪ Based on the key themes emerging from the analysis ▪ The goal is to explore the patterns emerging across informants Group interviews Focus groups or round tables ▪ Discussion guide with mostly open-ended questions, the first few of which are warm-up questions to get participants talking. ▪ Meeting of 8-12 participants in a meeting room at a neutral location ▪ The room is equipped with one-way glass ▪ A technical room is installed behind this glass, the analysts are behind the glass ▪ A moderator leads the meeting ▪ The moderator asks the questions ▪ Ensures that everyone has their say ▪ Prevents one person from monopolising the discussion ▪ The discussion is recorded (audio and/or video), transcribed and analysed The survey method Surveys Data is gathered through a structured questionnaire given to a sample of respondents and designed to elicit and collect specific information from them. It’s the most popular way of collecting data for market researchers. Terminology Survey ▪ The method/technique used to collect quantitative primary data Questionnaire - The tool for conducting a survey Interview ▪ The moment of the data collection, when the participant fills out the questionnaire ▪ Yes, the term interview is also used for quantitative research! The Survey in brief Employs structured questionnaires ▪ Standardized questions ▪ In a fixed order Key characteristics - Simple to administer - Reliable data, easy to compare But not exempt from problems: ▪ Respondents are not always willing or able to answer, are unaware of their preferences, or the options might not fully represent their views ▪ Surveys are not easy to develop (wording, sequence of questions, etc.) Sampling  The Sampling Process Define the target Sample selection population Select the sample Type of sample technique Determine the sample Number of size participants Execute the sampling Recruitment process Nome Cognome o evento, data process 1. Define the target population The Population is the aggregate of all the “elements”, sharing some common characteristics, that comprises the universe for the purpose of the marketing research problem Elements are the ‘objects’ about which the information is desired. For example, all French fast fashion consumers; all individuals who have recently bought an electric car; all those who have switched from Apple to Android in the last 2 years…. Usually individuals (or respondents), but also households, companies, etc… - DINK couples (Double Income No Kids) - Haute couture houses; soft drink producers; SMEs in the textile industry The target population is selected according to the research objectives  From the population to the sample An entire population is rarely involved in marketing research Marketing research usually involves a SAMPLE of the population A subgroup of the elements of the population selected for participation in the study 2. Select the sampling technique Each individual of the population has an equal probability of selection. Probabilistic Every individual is selected independently of every other individual and the sample is recruited through a random procedure Sampling Technique Selection is not based on the use of chance selection procedure. Sample Non- group members have to be selected Probabilistic on the basis of accessibility or personal judgment of the researcher. Non probabilistic samples Convenience Sampling: individuals are selected because they are easy to be reached by the researcher or because they are at the right place at the right time. Non probabilistic samples Judgmental Sampling:  Selection of individuals involved is based on the judgment of the researcher or individual’s expertise. This sampling technique is most effective when only a limited number of individuals possess the trait that a researcher is interested in.  Examples:  Professional buyers (e.g., engineers)  Opinion leaders  Bloggers Non probabilistic samples Snowball Sampling:  This sampling method involves primary data sources nominating another potential primary data sources to be used in the research. In other words, additional subjects are recruited via chain referral. 3. Determine the sample size What you need to consider: 1) Nature of the research and objectives 2) Size of the population 3) Analysis to be conducted 4) Resource constraints  Check previous similar studies to determine the sample size  Sample size should be determined based on statistical criteria… but often these contrast with business realities.  In most cases, a sample size of 300-400 respondents for consumer studies is more than acceptable. 3. Determine the sample size Definition of the sample size vs. final sample size Determined by statistical criteria Not able to reach the sample size Deletion of not completed interviews Number of completed  Anticipate refusal to Response interviews participate Rate = Number of eligible units in the About 60% door-to-door sample interview About 50% telephone interview About 30% mail survey 4. Execute the sampling process  How to recruit participants: - Mailing lists - Online recruiting (e.g., on the brand’s social media channels) - Onsite recruiting (in a mall, during events) - Marketing research providers - Online panels (e.g., Amazon Mechanical Turk) Panels  Professional panels are composed by individuals who agreed on being contacted for participating in marketing research. They are rewarded with monetary incentives.  https://www.mturk.com  Limits:  self-selection bias  Not representative  Sometimes, lack expertise How to administer a survey How to administer a survey Face-to-face interviews Telephone interviews Mail interview Electronic interviews  Face-to-face interviews ADVANTAGES High degree of At-home interviews Mall-intercept environmental control (no distractions) The interviewer can help in understanding the question More expensive Less expensive DISADVANTAGES Consumer scepticism Need to be No need to contact Interviewer bias arranged consumers (questioning, respondent beforehand beforehand selection) Need to train interviewers High percentage of Both types of F2F interviews can be done participants selected to paper-and-pencil or with the aid of a participate refuse to participate laptop/tablet  Telephone interviews ADVANTAGES Traditional Computer- assisted The interviewer can (CATI) clarify ambiguous or difficult questions Moderate level of control: the interviewer cannot see the environment but controls the questionnaire DISADVANTAGES  Faster o Questionnaires must be short o Consumer scepticism  Data are o Interviewer bias  the immediately presence of the interviewer available influences the answers o About 50% of participants selected to participate refuse to answer Mail interviews Questionnaires sent by regular mail to existing or prospect customers. It often involves panels of consumers that have agreed to participate in periodic mail questionnaires or product tests  80% response rate How to administer a survey / electronic interviews Electronic interviews Respondents autonomously answers to online questionnaires. Respondents can be reached through emails or when they ADVANTAGES DISADVANTAGES No missing data – forced Self selection bias: only answers visit a website. those participants who High flexibility & skip logic have an interest in the Fast and cheap topic take part to the Quick to develop and modify study Images and sounds can be Self-administered: no added clarifications Convenient – respondents can answer when they prefer Typologies of questions Main types of questions Open Vs. Closed Prompted Spontaneous Pre-coded Open-ended = box to be filled by the respondent Appropriate to measure brand awareness (e.g. top of mind) and adv content recall  Open vs Closed questions: the case of brand awareness Brand awareness is related to the strength of the brand in consumers’ memory, as reflected by their ability to identify the brand under different conditions. TOP- The first brand that comes into consumers’ mind when OF- asked about a product category MIND Ability to recall a brand spontaneously when Brand the product category is mentioned recall Brand Ability to recognize a brand by its package, recognition color, logo etc. when it is mentioned or shown to (or aided recall) the consumer Unaware of the Even when a brand is mentioned or a brand product is shown, the consumer is not aware of it Open vs Closed questions: the case of brand awareness Open questions are required: TOP- What soft drink brands are you familiar with? OF- When you think of soft drinks, which brands come to MIND mind? Please name all the brands of soft drinks you can think of. Brand Please name all the brands of soft drinks you can recall think of that you would bring to a party. Brand Closed questions are required: recognition Have you heard of (brand name)? YES/NO Are you familiar are you with (brand name)? (or aided YES/NO recall) Which of the following brands of soft drinks Unaware of the have you heard of? (list of alternatives) Do you remember having seen this brand? brand YES/NO Dichotomous vs multiple-choice questions “Have you purchased any “What smartphone did you purchased?” smartphone in the last 6 months?” o iPhone X o Yes o Galaxy o No o Huawei o …. Likert Scale Questions Most used rating scale to measure the degree of agreement (vs. disagreement) with each of the statement about the object under investigation. - Data are treated as interval - Easy to construct and administer - Easy for participant Likert Scale Questions Likert scale often uses multiple items that are then condensed in a single variable by calculating the average score.  Presence of reversed items to reduce the effect of the respondence tendency to answer with the end-scale option. Semantic Differential Questions 7-point rating scale where at the two extremes you find two bipolar labels. - Not suitable for all situations - Inclusion of reversed items - Need a pre-test of the adjectives (e.g., qualitative interviews) Likert vs. Semantic differential questions  Likert scales can be used to  Semantic differential measures measure: the participant position along a  Degree of continuum between two opposite agreement/disagreement extremes.  Degree of  For example: satisfaction/dissatisfaction  Very likely/very unlikely  Not at all/very much  Etc.  Likert scales are:  + flexible!  + easy to build  + the meaning of each option can be specified Continuous rating questions Respondents are asked to rate an object by placing a mark on a continuum. - Easy to build - Often used in online questionnaires  not suitable for paper and pencil questionnaires How many answer options? Except for dichotomous questions (e.g., 0/1), most other questions will require a greater number of measurement levels. For example, you could ask to rate the perceived quality of a product from 1 to 3, 1 to 5, 1 to 7, 1 to 9 etc. In most cases, it is preferred to use 5 +/2 measurement levels (i.e., from 3 to 7): Fewer measurement are preferable for telephone interviews, in case of low participant involvement, and for multiple-item scales. In these cases, it’s ok to have 5 or even 3 measurement levels. A greater number of measurement levels (>7) s more difficult to handle for most respondents. An odd or even number of answer options? Odd measurement levels (e.g., 5, 7, 9) Presence of a mid-point offers the possibility of being neutral But: always remember to add a If many respondent choose the mid-point level, ‘no option’/ ‘don’t know’ category data can be less useful Even categories (4, 6, 8, 10) Force respondents to express an opinion; Data can be more useful Scale: A term with many meanings (don’t get confused) ▪ Likert scales, semantic differential scales ▪ Types of questions ▪ A scale is also a set of items used to measure the same variable. ▪ For statistical reasons, it is preferable to include more than one item to measure a same variable. This will increase the scale reliability. ▪ An item is a single statement (see next slide) Examples of multi-item scales Visual appeal (Wang 2003) ▪ Using 3- or 7-point semantic differential, the scale measure the degree to which something (a product, a packaging, a website) is considered to be interesting and creative ▪ Scale items are: ▪ Monotonous / Fascinating ▪ Conventional / Creative ▪ Unremarkable / Impressive Attention to the Ad (Yoon 2015) ▪ A person’s expressed interest in the message content of an ad and the attention he/she paid to it is measured with three, seven-point Likert-type items. ▪ Scale items are: ▪ I was interested in what the ad had to say ▪ I paid close attention to the ad’s arguments ▪ I didn’t let myself get distracted from focusing on the message content. Creating a questionnaire The process of preparing a questionnaire Step 1: OBJECTIVE DEFINITION AND CONCEPTUALIZATION Step 2: DRAFTING THE QUESTIONNAIRE Step 3: TEST THE QUESTIONNAIRE The process of preparing a questionnaire Step 1: OBJECTIVE DEFINITION AND CONCEPTUALIZATION Define the topics to be investigated (based on managerial input, but also on the result of exploratory studies and qualitative research) List the variables Think about the statistical analysis to be conducted: mere description or causal research? Typical survey goals (1): Measure consumer behaviors and attitudes Example – McKinsey Survey: Consumer sentiment on sustainability in fashion Participants to the survey reported their behavior change during the Covid-19 crisis: - “I have made significant changes to my lifestyle to lessen my environmental impact” - “I have started to go out of my way to recycle” - “I have gone out of my way to buy products in environmentally friendly packages” Typical survey Goals (2): Quantify and generalize insights from a qualitative study Example. During a qualitative study based on a small number of in-depth interviews, a company finds out that the main causes of dissatisfaction towards a products are: higher perceived price (compared to product alternatives); unattractive product design; reduced assortment; difficult-to-recycle packaging; long shipping time. It is now time to quantify the relative importance of these sources of dissatisfaction, and their possible impact on the intention to switch to another supplier, with a quantitative study with a representative sample of consumers. From exploration to testing Often, after a qualitative study Qualitative study / desk or desk research, the initially research insights are followed by quantitative research used to: Measure a phenomenon Insights on the Test its presence in the market phenomeno n Understand its relationships with other variables Test hypothesis Quantitative phase Survey Goals (3): Identifying the positioning of a brand compared to competitors Example – Perceptual maps of beverage sweetener You need to measure these dimensions in your questionnaire (perceived naturalness, perceived health) for your brand and its main competitors. To have significant results, the sample will need to be representative of your market segments Survey Goals (4): Identifying consumer profiles (for market segmentation) For the USA, the VALS Survey is a multiclient, standardized questionnaire BUT each brand can create their own questionnaire with variables of their interest and use the data to identify market segments Do it at home: http://www.strategicbusinessinsights.com/vals/surveynew.sht ml Survey Goals (5): Understand Customer Satisfaction Example – University course evaluation Students evaluate each course they attend based on the following dimensions: o Course objectives – “Course objectives are clearly expressed” o Knowledge and skills acquisition – “I developed new knowledge and skills” o Teaching clarity – “The professor teaches in a clear and structured way” o Ability to stimulate students – “The professor stimulated my interest” o Interactivity – “The professor encourage interactivity during the sessions” o Resources – “The resources (exercises, cases, lectures, etc.) are adequate to facilitate the learning” What you can do with surveys: Descriptive research For each question in the questionnaire, means or frequency distributions are calculated. For example, you can obtain consumer data about: ▪ Attitudes towards a brand ▪ Satisfaction about products ▪ Intention to buy ▪ Word of mouth behavior Data are typically reported with tables, pie charts, and histograms. What you can do with surveys: Causal research With causal research, you can use more complex statistical methods to discover relationships between variables… ▪ A more positive attitude towards the brand generate a greater intention to buy … or test differences between subgroups of participants… ▪ Women vs. Men have different attitudes towards the brand … or describe consumer profiles ▪ Consumers high in self-monitoring and who love the brand will generage more word-of- mouth Dependent vs. Independent variables Dependent variables, DVs (also called response variables) – are variables whose value depends on or responds to other variables. Typically, the dependent variables (DVs) are the results the marketer wants to achieve. Typical DVs in consumer research include: ▪ Attitude towards the brand ▪ Intention to buy ▪ Willingness to pay ▪ Customer Satisfaction ▪ Customer Loyalty Independent variables (IVs) are variables that influence or determine DVs. An DV can be influenced by multiple IVs. Typical IVs in marketing and consumer research include: ▪ Elements of the marketing mix (e.g., price, perceived product quality) ▪ Consumers’ personal characteristics (e.g., personality traits, perceived importance of a product or service). At the end of the step 1 You should have clear goals, based on the results of previous exploratory studies You should have listed the main variables that are relevant for your study If you want to carry out causal research, you should have defined the relationships between dependent and independent variables. For example, you could explore if: - Perceived product quality has a positive impact on customer satisfaction (CS) - Perception of expensiveness has a negative impact on CS - More satisfied consumers are more loyal The process of preparing a questionnaire Step 2: DRAFTING THE QUESTIONNAIRE Define the order of the topics and the structure of the questions Prepare filter questions Formulate the questions Drafting the Questionnaire Before developing the questionnaire, it is important to map the flow of questions. A flow diagram (see next page) can help to ensure that all the important topics are covered. Include filter questions: these are questions that aim to filter survey respondents (i.e., if they answer in a specific manner to a question, they are directed to specific parts of the questionnaire). Example: do you own a home? Yes  they are home-owners --> go to questions 2 to 6 No --> they are tenants  go to question 7-10 Example 2: have you bought fast fashion products in the last 12 months? Yes  they are part of our sample  continue with the survey No  they are not part of our sample  go to a thank you page, survey is over  Online booking platforms usage Yes NO Type of hotel booked Why not used? Frequency of using online platforms for Other forms of booking booking adopted Whether used to book Any None an apartment Whether used to book an apartment NO Yes NO Yes Why apartment preferred for certain Why apartment Classify and booking preferred for certain close booking Comparative rating of forms of online booking, convenience, ease etc. Classify and close Questionnaire structure Introduction Filter questions Questions Organized in sections according to the topic Linking sentences Conclusion Thanking participants Disclosure about the aim of the study Questionnaire introduction Dear participants, we would like to invite you to participate in this research project. We are a group of students of the emlyon business school, and our professor will fail us unless a sufficient Do always include number of respondents will participate in our survey. in the introduction: The information collected is completely anonymous and will - Salutation be used solely for the purpose of this research project. - Thank Please note that we are interested in your personal opinion - there are no right or wrong answers. The survey takes 5 participants minutes to complete. - Length of the questionnaire Thank you very much for your valuable participation - Informed consent By clicking on the 'continue' button, I agree on taking part to - Info about data this study. use - privacy  Suggestions ▪ Be parsimonius with questions (if the questionnaire is too long, the completion rate will decrease) ▪ General and easy questions at the beginning  funnel strategy (from the most general to the most specific questions)  initial open ended question  the initial question should be meant at increasing respondent motivation and trust – it can also be unrelated to the topic under investigation  sensitive questions not at the beginning ▪ Justify why you need some information (especially for sensitive questions)  Suggestions ▪ Consider the context where the interview takes place ▪ Minimize participants effort and fatigue  clear instruction  short questionnaires ▪ Clearly defined questions – who, what, when, why and how ▪ Check the style and readability  Avoid ambiguous questions and words ▪ Ambiguous questions ▪ they can be interpreted in different ways by respondents, making data unreliable ▪ Ambiguous words ‘never’ ‘less than once’ ‘occasionally’ ‘1-2 times per ‘sometimes’ VS. week’ ‘often’ ‘3-4 times per ‘regularly’ week’ ‘5-6 times per week’ ‘everyday’ Good example Good example Bad example ▪ No instructions ▪ No labels to understand what 1-5 means The most important suggestion of all: If you can, don’t invent questions, but use validated scales Use validated scales Consumer researchers have developed statistical methods to validate measurement scales, ensuring that they have good psychographic properties (i.e., they actually measure the psychological constructs they are developed to evaluate). Bachelor students lack the training to develop good scales. As a result, their questions result have limited reliability. The good news is that for most variables that you might want to include in your questionnaires, validated scales already exist. They are easy to find with google scholars and the emlyon library. Just search for published articles using the same variables, check the methodology sections and use the same reported items in your questionnaire. Use validated scales Use validated scales You can use google scholar to search for scales used to measure variables of interest. Use validated scales The process of preparing a questionnaire Step 3: TEST THE QUESTIONNAIRE Make sure that the questionnaire works properly Pilot test Pilot Test Informal pilot: Ask a small Do the questions sound right? convenience sample to Do the respondents understand the answer and then collect their questions? Can they answer the questions? reactions to the questionnaire Can the respondents understand the Accompanied pilots: you instructions? are present when the pilot Does the questionnaire flow properly? test is done, and the How long does the questionnaire take? participant tells you how he/she feels Have mistake been made? Large-scale pilots: Large Does the technology work? sample (for large studies) to check scale reliability Reading: A concise guide to market research (link on BrightSpace) Chapter 3, Data (pp. 25-45) – all chapter except paragraphs 3.6-3.7 After reading this chapter, you should understand: How to explain what kind of data you use; The differences between primary and secondary data; The differences between quantitative and qualitative data; What the unit of analysis is; When observations are independent and when they are dependent; The difference between dependent and independent variables; How to set up different sampling designs; How to determine acceptable sample sizes. To pass the individual exam, you should be able to answer the review questions at pp. 44 (please disregard questions 4, 6, 7, 8, 9), learn and recall the meaning of the keywords employed. Pay particular attention to figures, tables, and formulas. Chapter 4, Getting Data, only paragraphs 4.4.2 (Designing Questionnaires, pp. 60-77) and 4.5 (Basic Qualitative Research, pp. 78-81). After reading these paragraphs, you should understand how to set up a basic questionnaire and basic qualitative research. These paragraph contains important notions that you will apply to your assignment n. 2. To pass the individual exam, you should study very well the notions contained in these paragraphs. Reading: Asking questions with reflexive focus: A tutorial on Designing and Conducting Interviews Your group assignment will not be based on qualitative research to reduce its complexity and the time required to complete it. During the second semester’s GBBA field study, you might find yourself in the need to carry out qualitative interviews. On BrightSpace, you will find this tutorial as reference. Studying it is not required to complete your group assignment or pass your individual exam.