Market Research Notes PDF

Market Research 1. Organiza*on of a Market Research What is Market Research? The func3on that connects the customer to the marketer through informa3on. Enhance customer sa3sfac3on. Introduc3on of a new product in the market Customer Sa3sfac3on - find what consumers want/need and produ...

Market Research 1. Organiza*on of a Market Research What is Market Research? The func3on that connects the customer to the marketer through informa3on. Enhance customer sa3sfac3on. Introduc3on of a new product in the market Customer Sa3sfac3on - find what consumers want/need and produce/offer in a way beEer than the compe3tors. Customer sa3sfac3on is the company’s main goal. The role of Market Research in Decision Making: I. Define the marke3ng problem. II. Iden3fy relevant and necessary informa3on. III. Collect data. IV. Process informa3on in a useful way for the management. V. Find alterna3ves to solve the marke3ng problem (present solu3ons). Goal: produce informa3on about customers that allows the market manager to decide beEer, enhancing customer sa3sfac3on. We should do a Market Research when… 1. The informa3on we have is not enough to answer our ques3ons (when we want to search for more informa3on). 2. We can use the research results, whatever they are. 3. There is 3me to use the collected data. A#en&on: Market Research can be easily manipulated or used for the wrong purposes! Let’s see one example… The 1st op3on is beEer, since we have 2 posi3ve answers and 2 nega3ve answers (balanced scale). The 2nd op3on can lead to biased results since it has 3 posi3ve answers but only 1 nega3ve answer (unbalanced scale). Situa3ons where we can use unbalanced scales: if the goal of the research is to highlight posi3ve aspects of a product/service, using this type of scale can help emphasize posi3ve feedback. It is suitable when a company wants to showcase the strengths of its offerings (however it is not recommended). Market Research Process 1. Problem Defini3on (Planning) Sets the research objec3ves. Defining well a problem is crucial to design a research that will produce relevant informa3on. The marke3ng problem is almost always related with change (planned or unexpected) of consumer behaviors, new demands, consumer expecta3ons, among others. If the problem is not well defined it leads to wrong decisions! Marke3ng Problem: o Introduce a new product in the market? o Raise the price of a certain product? o Develop packaging for a new product. vs. Researcher Problem: o Determine consumer preferences and purchase inten3ons of the new product. o Determine sales impact of several price changes. o Evaluate impact on sales of different packaging alterna3ves. Decision alterna3ves: o Problem: phrase for the need for informa3on (is it viable to open a bowling salon?). o Decision Alterna3ves: hypothesis that contribute to solve the problem (open or not open the bowling salon). o Decision Criterion: to separate the decision alterna3ves (inten3on to aEend the bowling salon). o Decision Rule: how to use the criterion to choose the best alterna3ve (if at least 40% of individuals demonstrate inten3on to aEend the bowling salon, it will open). 2. Research Design (Design) Define the informa3on we need. Type of research: exploratory or conclusive. Define the target popula3on. Define the sampling frame. Define the process of sample selec3on. Sample design and sample size. Design and test the methods to gather informa3on. o Ques3onnaire (ask) or observa3on (usually observe behaviors that we aren’t aware of)? o Ques3ons with open or closed response? o Objec3ves of the study should be explained to the respondents? (explain to have more realis3c and honest answers or not provide many details to engage people and not be boring/lead to biased results). Plan for data analysis: qualita3ve or quan3ta3ve?  Collect informa3on should ideally be accurate, updated, enough, available, and relevant (this is almost impossible). How to choose the most suitable research techniques? Understanding respondents. Nature of respondent: Nature of issue under inves3ga3on as perceived by the respondent, and context/environment where respondent is ques3oned. 3. Data Collec3on (Performance) Qualita3ve/quan3ta3ve research (ques3onnaire). Test the topic guide/ques3onnaire (pilot test). Selec3on and training of field staff. Fieldwork (interviews). Quality control. 4. Data Analysis (Performance) To collect useful data, it is necessary to analyze the respondents thinking about the marke3ng problem of the research. Edi3on – check if data is complete and consistent. Tables – summarize the data in tables. 5. Report and Recommenda3ons (Performance) The research report with a summary of the results and conclusions is submiEed to management. Most important document for the project evalua3on performed. Clear and precise. Research Designs: o Exploratory: - to beEer understand the phenomenon under study. - explore sensi3ve issues. - formulate hypothesis. - explore ways of communica3ng a new product. o Conclusive: Descrip3ve: - to describe/measure the features of the market. - define consumer profile. - adver3sing studies describing audience profiles. - image studies for constant percep3on. Causal: - to test specific hypothesis and examine cause-effect rela3onships. - iden3fy which variables are the causes (independent variables) and which are the effects (dependent variables) of a market phenomenon. - test hypothesis. Exploratory: o The informa3on needed may not be well defined. o Research process is flexible and unstructured. o Small sample (diversity is important!). o Data analysis can be qualita3ve or quan3ta3ve. o Individual in-depth interview: conducted by an interviewer, 1 single respondent semi-structured topic guide (flexible), and dura3on between 30 min and 1h30. o Focus group: conducted by a moderator, small group (6 to 10), flexible topic guide to support, and dura3on between 1h30 and 2h. Conclusive: o The informa3on needed is clearly defined. o Research process is formal and structured. o Sample is large and aims to be (sta3s3cally) representa3ve. o Data analysis is quan3ta3ve. Funnel of the detec3ve: o Problem -> Exploratory Research -> Hypothesis -> Descrip3ve Research -> Causal Research -> Solu3on o I.e.: This feature of the service will increase customer sa3sfac3on and inten3on to buy. Primary Data vs. Secondary Data: Primary Data: o Data originated by a researcher for the specific purpose of the present problem. o Can result from an exploratory or conclusive research. o May include explora3on of databases. Secondary Data: o Data that have already been collected for other purposes than the present problem. o Access to informa3on already produced. o Analysis of literature/case studies. The main objec&ve is to use all internal and external resources to develop a deep understanding of the market of that product/service. Advantages of secondary data: o Saving 3me. o Low cost. o Less effort in collec3ng (easy access). o Some types of informa3on can only be obtained through secondary data (i.e.: census). Limita3ons of secondary data: o Data collected for other purpose (can’t be 100% accurate).  o Lack of control in the data collec3on. o Possible lack of data quality. o May not be in the required format. o May be outdated. o May not meet the needs of the study. Analysis of available secondary data is a prerequisite to the collec&on of primary data. Important: evaluate secondary data before use it, regarding: I. Research design: size/nature of the sample, response rate, ques3onnaire design, rules of data collec3on, data analysis). II. Error and accuracy: is the data accurate enough for the purposes of our study? III. Data of data collec3on: value of secondary data decrease as they become dated. IV. Purpose for which the data were collected (e.g..: the target popula3on may be different). V. Content of the data (e.g.: units of measurement). Secondary data can be: o Internal: data generated within the organiza3on for which the research is being conducted. We should search for it first. o External: data generated by sources outside the organiza3on (e.g.: traded or published data). It is more difficult to access, expensive, and difficult to evaluate the accuracy. Search for it secondly. Target Popula3on: all the units (individuals, households, organiza3ons) to which we want to generalize the market research results. Iden3fica3on of the target popula3on (can’t be ambiguous): o Define precisely. o Consider the objec3ves. o Consider alterna3ves. o Know the market. o Define the scope (e.g.: geographic). o Iden3fy the sampling unit (household to which the client belongs) and the observa3on unit (client of a bank). Sample: units of the target popula3on that are chosen to be observed/interviewed. Sampling Frame: list of all the units that belong to the target popula3on. It is the list from which we select the sample. Ideally, it is an exhaus3ve list and without duplica3on of the units of the target popula3on. If we a have a sampling frame, the process will be more efficient since we have a way to contact the clients, and addi3onally reduce the non-sampling errors. However, it is not always easy to gather this.  If we consider that one is not good enough, we can combine several sampling frames, namely from telephone books, databases, mailing list. This way, we will obtain a reasonable coverage of the popula3on, but this process can introduce duplica3ons. We should maintain them updated (can become outdated). There are popula3ons that is very difficult/impossible to get a sampling frame. It can be very difficult to guarantee the consistency of our sample, and normally there are sample elements that are not in our target popula3on. To avoid this, we should implement selec3on ques3ons, which will guarantee that elements that do not belong to our target popula3on, are not part of our study. We should always carefully explain the target popula3on. May be hard to get. What to do when there is no sampling frame? It is necessary to specify a process to find the sampling units. Methods based on internal informa3on: client registra3on process, customer care service, promo3ons, customer loyalty program. Methods based on external informa3on: I. Select a sample of customers from the target popula3on using the genera3on of random numbers (RDD - random digital dialing). A#en&on: we can use RDD with fixed networks, but not all houses have them (those have a null probability of selec3on), so we use mobile networks (may create duplica3ons). II. Random selec3on from telephone lists. A#en&on: avoid coverage problems by using both mobile and/or fixed networks (can combine them) but may also create duplica3on of contacts. III. Intercep3on in places of consump3on. A#en&on: can lead to bias since it only considers people who visit places of consump3on. Look at profile’s diversity and select places carefully to have interviewees who have socio-demographic characteris3cs of the target popula3on (≠ days, hours, places). Know the market first and combine with two-stage sampling. IV. Use of external databases. A#en&on: we should be careful with the quality when we buy a database, as it can result in unknown biases (it must not be used for conclusive analysis). V. Database regarding groups of observa3on units. A#en&on: some3mes we don’t have a list of the individuals, but we may have the list of the clusters (groups). We should use two-stage sample (observe a sample of individuals in some clusters and use qualifica3on criteria to choose from them). Important: when there is no sampling frame for a specific popula3on, a solu3on is to combine sampling from telephone lists and RDD with qualifica3on criteria (difficult in popula3ons with low incidence). 2. Data Collec*on: Ques*onnaire and Scales Ques3onnaire: structured measuring instrument for obtaining data from respondents. o Ques3ons follow a previously established order. o Respondents can choose between a fixed set of possible answers. o 1st we decide the way we will implement the interview and then we design it. o Always pay aEen3on to colors, leEers (size and font), pictures, and order of the ques3ons. o Survey ≠ Ques3onnaire: the survey englobes all the process/study and the ques3onnaire is a list of ques3ons that can be applied in several forms. Possible types of response scales: o Closed (e.g.: sa3sfac3on level) - we should use this one, since it normally is easier to analyze and answer. o Semi-open (e.g.: reason of choice)- we have fixed op3ons, but respondent can also give other answers if needed. Easier scale to analyze compared to open ques3ons. o Open (e.g.: sugges3ons)- not the best one. We shouldn’t use it for variables like age (use intervals instead). Normally used it just in the end for sugges3ons. The order of the ques&ons is important, and we should not change it because it will influence the answers. We want all individuals to have the same influence when answering, to not have bias. Advantages of Ques3onnaire: o Simple to apply. o Data obtained is consistent (responses limited to the alterna3ves). o Reduces the variability of the results (caused by differences between the interviewers). o Facility in coding, analysis, and interpreta3on of the data. Disadvantages of Ques3onnaire: o Inability (lack of knowledge about the topic) or refusal to answer (sensi3ve topics). o Difficulty in measuring opinions and feelings since ques3ons only have fixed alterna3ves. o Difficulty in formula3ng correctly the ques3ons. The ques&onnaire design is an art, but it is always subjec&ve… Basic rules for Ques3onnaire design: o Formulate ques3ons that respondents can and want to answer. o Mo3vate and engage the respondent in the interview.  o Minimize the response error. What the respondent may want: o Tangible reward o Confiden3ality o Interes3ng subject/experience o Personal and social benefits o Credibility of the research organiza3on o Empathy and trust in the interviewer What the researcher wants (from the respondent): o Honesty o Follows the instruc3ons o En3re ques3onnaire completed o Thinks about the issues before answering o Gives clear answers o Cooperates with the methods and aims of the study o Says good things about the research process We should have a deep knowledge regarding the study. There are some steps to follow: I. Specify the informa3on needed o Ensure that the informa3on obtained fully addresses all the components of the problem. o Design a data structure to help define how the collected data will be analyzed. o Have a clear idea about the characteris3cs and mo3va3ons of the target popula3on. II. Specify the type of interviewing method o Personal: longer, complex, varied ques3ons and has a form of dialogue. o Telephone: short, simple ques3ons and has a form of dialogue. o Mail: short, simple ques3ons and has detailed instruc3ons to follow. o CAPI (Computer-Assisted Personal Interviewing), CATI (CA Telephone Interviewing) and CAWI (CA Web Interviewing): complex filters between ques3ons and randomiza3on of ques3ons. The order can influence the answers. For example, a ques3on about the price or sa3sfac3on can influence the mood of the interviewee. Usually, we should have a unique order, however some3mes we change it to minimize the bias because some ques3ons can influence the next ones. Therefore, we ask in a different order to “balance” its influence. III. Determine the content of the ques3ons: o We should just ask if we need to know the answer. Every ques3on should contribute to the informa3on needed.  o Some ques3ons have another specific purpose, such as: ini3al ques3on(s) to establish involvement when the topic is sensi3ve, filter ques3ons to disguise the purpose or sponsorship of the study (e.g.: just for students with age between 18-25 years), and ques3ons to assess reliability/validity (e.g.: iden3fy difficul3es/confusions in some ques3on of the ques3onnaire). o Moreover, some3mes we need to perform several ques3ons to obtain the wanted informa3on (ques3ons with double informa3on). E.g.: Do you think Coca-Cola is a tasty and refreshing drink? - Normally we shouldn’t apply this kind of ques3ons since we can agree only par3ally, and not give a 100% accurate answer, but we might not have another op3on. IV. Overcome the inability and unwillingness: o Is the respondent informed? To guarantee we should apply filter ques3ons and/ or give and “I don’t know” op3on. o Can the respondent remember? Give extra informa3on to help, like a list of the op3ons. o Is the respondent able to ar3culate? Help with images, maps, or alterna3ve descrip3ons. o Always minimize the effort required to answer. o Not ask ques3ons out of the context. o If we need to ask sensi3ve ques3ons, they should be asked at the end, mixed with easy ques3ons (hide it in a group of ques3ons the respondents are willing to answer), and provide responds categories (e.g.: when we ask about income). o Legi3mate ques3ons. o In the beginning, we should perform selec3ve ques3ons to check if they are part of the target popula3on. o Some3mes we want to ask more personal informa3on so we should put those ques3ons in the end and not mandatory, but there can be excep3ons like when we have a sensi3ve topic, we should introduce these ques3ons in the beginning (always explain the purpose). V. Choose the ques3on structure o Unstructured Open-ended ques3ons: not use them a lot, but if we use place them at the end. - Advantages: wide variety of responses, respondents may express themselves, and less bias in the response. - Disadvantages: can lead to interviewer bias (don’t explain the ques3on clearly), costly and 3me consuming in register and analysis, and not very suitable for self-administrated ques3onnaires (hard analysis). o Structured Mul3ple choice ques3ons: the order of the ques3ons can influence the answers, response alterna3ves should be exclusive/dis3nct, number of alterna3ves between 5 and 7, we can use an open answer (Other?  Which?) and a “I don’t know” op3on, we can apply the rota3on of alterna3ves, make sugges3ons or not? - Advantages: interviewer bias reduced (we have fixed op3ons) and facilitates the register and analysis of the data. Dichotomous ques3ons: with 2 op3ons of answer (Yes/No and Agree/Disagree), don’t suggest, and usually include the “I don’t know” op3on. Scales: - Types of scales: nominal, ordinal, interval, and ra3o. VI. Choose the ques3on wording o Define the issue correctly (singular/plural, who, what, when, where…). E.g.: Which brand/brands of shampoo have you personally used at home during the last month? In case of more than 1 brand, please list all the brands. o Use accessible language to guarantee that all interviewees understand the ques3on. o Avoid ambiguous words (use concrete values instead of what we define as never, occasionally…). o Avoid ques3ons that induce the response (ask generally without sugges3ons). Don’t ask “Is Colgate your favorite toothpaste?”, ask “What is your favorite brand of toothpaste?” instead. o Avoid implicit alterna3ves. Don’t add “…or do you prefer another alterna3ve?”. Instead, use a mul3ple-choice ques3on with all the alterna3ves. o Use posi3ve and nega3ve statements. E.g.: to agree/disagree: “The store is well located” and “It is difficult to park”. VII. Organize the ques3ons in proper order o 1st: Qualifying ques3ons- Does the poten3al respondent belong to the target popula3on? o 2nd: Basic informa3on - Related directly to the research problem. o 3rd: Classifica3on informa3on- Socio-economic and demographic characteris3cs. o 4th: Iden3fica3on informa3on (op3onal, not always used)- Name, telephone, address. o Open ques3ons o Difficult ques3ons (sensi3ve, embarrassing, complex, dull/monotonous. o Logical order: all ques3ons about a certain topic should be together. Branching/filter ques3ons guide the interviewer/respondent directly to a part of the ques3onnaire depending on the answers given. Respondents should not an3cipate the next ques3on (spontaneous and unbiased answers).  vs. o General vs. Specific ques3ons: if we want to ask about a global sa3sfac3on we need to specify if it is about the last interac3on, on average. When we analyze the answers, we need to have in considera3on the order because it influences the answers. - Funnel approach: general ques3ons precede specific ones. 1st: What is your overall sa3sfac3on? 2nd: What is your sa3sfac3on with the following aspects…? VIII. Iden3fy the form and layout o Divide into several parts/sec3ons o Ques3ons should be numbered o Pre-code the response alterna3ves (1 for strongly disagree, 2 for disagree and so on) o Ques3onnaires should be numbered? Give a code for each person if we want to guarantee that they just respond once. Could be necessary to write in the ques3onnaire a region or interviewer code IX. Reproduce the ques3onnaire o Professional appearance o Book (folded A3) for long ques3onnaires o Don’t separate a ques3on on different pages o Response alterna3ves in ver3cal column o Don’t crowd the ques3ons together o Don’t use a too small font o Instruc3ons close to the respec3ve ques3on o Colors and how many colors we should use (e.g.: use green for posi3ve and red for nega3ve) X. Pilot test o Small sample size (15 to 30 respondents)- in the project we only asked to 5 o Respondents should belong to the target popula3on o Use several interviewers o Researcher should conduct some interviews o We can ask the respondents to express their opinion/experience about the ques3onnaire o Ater the revision of the ques3onnaire, conduct another pilot test on a different sample o Responses obtained are coded and analyzed  o We should test the ques3ons: content, wording, meaning, difficulty to response, instruc3ons (clear or not), variability of responses (mutually exclusive), and interest and aEen3on of the interviewee. o Also test the en3re ques3onnaire: sequence, flow, filters and jumps - between ques3ons, dura3on, form, layout, and interest and aEen3on of the interviewee. Scales of Measurement: Nominal Scale: o Numbers assigned to iden3fy and classify objects o Classes/categories are exhaus3ve and mutually exclusive o No order o Only possible arithme3c opera3on: coun3ng the elements of each category (percentages, mode) o E.g.: Gender: 1- Female, 2- Male Ordinal Scale: o Numbers tell the rela3ve posi3ons of the objects, but not the magnitude of the differences between them o Ranks (preference/importance) the objects with respect to a common variable o Allows to determine whether an object has more/less a characteris3c than some other object, but not how much more/less o Arithme3c opera3ons are limited (median) o E.g.: Rank car brands according to your preference, from 1 to 4 where 1 is your favorite Interval Scale: o Use numbers to rank objects such that numerically equal distances on the scale represent equal distances in the characteris3c being used o Equal interval between scale values o Differences between object can be compared o Can apply almost the en3re range of sta3s3cal opera3ons (mean, standard devia3on, frequencies) o E.g.: Temperature (Celsius scale), Measurement of autudes/opinions (sa3sfac3on and loyalty), “I like listening to music while I study” - Agree (1)/Disagree (5) Ra3o Scale: o Interval scale but with a fixed zero point o Possible to compare absolute values and to compute ra3os of scale values o Possible to know how much an object has more/less of a characteris3c than another object o E.g.: Income, sales, costs, weight, distance We can also have… Compara3ve Scales:  o Direct comparison of objects o Small differences between objects can be detected o Respondents choose objects according to their preferences o Results have ordinal proper3es o Results cannot be generalized to other objects that were not used in the comparison Paired Comparison Scale: o Select one of the two objects (e.g.: select the one we prefer) o Useful when there are few objects o AEen3on to the order o Can turn into rank order scale o N brands, so n(n-1)/2 comparisons Difficult to understand Easier to understand Rank Order Scale: o Rank a list of characteris3cs (e.g.: by importance) o Easy to understand o Results are just ordinal data Constant Sum Scale: o Divide a certain number of points among some characteris3cs (more important, more points) o Advantage: very discriminant o Difficult to answer o Some3mes the sum is not equal to the total points o Can be every number (may not be 100 points) Non-compara3ve scales: o Respondents only evaluate one object at a 3me  o There is no comparison between objects, nor with some specific standard (ideal brand) Itemized Ra3ng Scales: o There is a number/brief descrip3on associated with each category (all of them or only some) o Decision to Make: 1. Number of categories (several factors should be accounted): - Knowledge of the respondent - Respondent’s involvement with the subject - Method for data collec3on - Nature of the object to be measured - Sta3s3cal analysis to apply 2. Balance scale: same number of posi3ve and nega3ve response alterna3ves 3. Odd or even scales: with or without a middle point 4. Forced or non-forced choice: use or not use the “I don’t know” op3on? We should not use it in all ques3ons 5. Nature and degree of the verbal descrip3on: strength of the poles (ends) and number of categories with a label (very dissa3sfied vs. dissa3sfied and very sa3sfied vs. sa3sfied) 6. Physical form: we use smile face scale if we have kids in our sample or if we want very quick/easy ques3ons, for example in a coffee shop or at the airport. Also choose ver3cal or horizontal scales, boxes or lines, posi3ve or nega3ve numbers, colors, font, and use or not the smiling face scale. Likert Scale: indicates the degree of agreement/disagreement of each of a set of statements about the s3mulus object. We analyze the results item by item, and sum all the items to get a total for each respondent - Total = 4 - A respondent has the most favorable autude towards the health insurance with the highest score) Seman3c Differen3al Scale: measures respondents' percep3ons about a par3cular object. It presents pairs of opposing adjec3ves, and respondents are asked to posi3on the object between these extremes, indica3ng their degree of preference/evalua3on.  - Seven-point ra3ng scale - Central point is neutral - Alternate nega3ve adjec3ves between the right and let side - Increments on the scale are considered equal (can calculate means: controversial since this is a characteris3c of interval scales) Stapel Scale: it has only one pole, there is no neutral point and usually is presented ver3cally. 3. Data Collec*on in a Quan*ta*ve Research Descrip3ve Research o Interviews - Telephone (CATI and Tradi3onal) - Personal (CAPI and Tradi3onal: In-home/In-office and Street/Intercep3on) - Postal (Tradi3onal and Electronic: CAWI and E-mail) Telephone Interview o There is only verbal contact, so we can’t use a ques3onnaire with images, and need to use simple scales to guarantee that the interviewee understands it all. o May cover a large geographical area quickly o Tradi3onal: Paper ques3onnaire o CATI: Electronic ques3onnaire - The computer dials the telephone number - Interviewer reads the ques3ons that appear on the screen and records the answers - The database with the answers is automa3cally prepared for analysis - The scheduling is automa3c - We should report the mean dura3on, number of aEempts, number of interviews, 3me to the interviewer - Dura3on of the interview is reduced, and the quality should increase - We can supervise by recording the interviews (all or some), having mystery clients from the company or a supervisor in the room - Important to consider: Þ Selec3on of telephone numbers: predefined list, phonebook or RDD Þ Introduc3on: tell the name, dura3on of the interview, and in some cases the topic of the study Þ Define 3me of the day for inquiry: morning, night… Þ Schedule the day and 3me convenient for the respondent  Þ Report about the interviews Þ Result of the call: if the phone is busy or no one answers we should call again, but if the call is answered by another person (doesn’t belong to the sampling frame/target popula3on) we should try talking to the person we want. If no one answers, we try another 3me of the day Personal Interview o Conducted in person o Know the place where the interview is going to proceed (should be a quiet) o Direct contact between the interviewer and the interviewee o Context (e.g.: psychological state of both, place and 3me of the interview) influences the process of data collec3on o More expensive and 3me-consuming approach o Methods: - Door to door interview (in-home): nowadays just perform it if it’s a scheduled interview - Execu3ve interview (in-office): people can be not comfortable answering, because of the presence of bosses, co-workers, among others - Intercep3on technique in place of consump3on (store/street) and when buying: only for buyers of that store - Self-administered ques3onnaire: can be explained and given in paper and can be anonymous or not. Allows interview more people in less 3me o Confiden3al vs. Anonymous: in confiden3al we can’t share but we can iden3fy (we can know) and in anonymous we don’t have informa3on to iden3fy the person (we can’t know) o Steps to follow (regarding interviewers’ rate of refusal): - Recruitment of interviewers - Training of interviewers - Defini3on of quality control procedures for conduc3ng the interviews and recording data: guidelines to ensure reliability, validity and accuracy of the data collec3on process - Supervision of interviewers Postal Interview o No verbal contact (no interviewer) which contributes to not have bias due to the way ques3ons are made. However, there is no one to answer to any doubt o Cheapest way to collect data o Can be sent by mail (electronic/tradi3onal) or posted on a website o Requires excellent iden3fica3on of the sample before the collec3on o Tradi3onal mail: decide on send and return envelope (RSF), presenta3on leEer, shape, size and color of the ques3onnaire, number of submissions, and incen3ves o Electronic mail (e-mail or CAWI): target popula3on must have access to the internet To choose the method of data collec&on… Factors influencing the choice: o Target popula3on (e.g.: geographically) o Format of the ques3ons (e.g.: have pictures) o Content of the ques3ons (e.g.: if it is a sensi3ve topic, people normally feel more comfortable answering alone) o Response rate o Available infrastructures o Cost o Dura3on of individual collec3on o Control of the fieldwork o Quan3ty of data o Perceived anonymity of the respondent o Poten3al for interviewer bias (opt for postal interview) Ethical issues in data collec3on: o Use surveys for purposes of sale or to get names and addresses for direct marke3ng o Deceive people about the objec3ves of the study o Lie about the dura3on of the interview o Lie about the incen3ve to get par3cipa3on (rewards) o Use hidden recorders to record the answers (we should inform we are recording to not viola3ng people’s privacy) o Not to say that there will be follow-up contacts o Codes of conduct and data protec3on: - ESOMAR: encouraging, advancing and eleva3ng market research worldwide - Portuguese Data Protec3on Authority: supervise compliance with the laws/regula3ons in personal data protec3on, and respec3ng human rights - APODEMO: promote trust in market research and contribute to the establishment of high standards of quality and ethical, by defining codes of conduct Reasons for not responding to an interview: 3me, age, interest, sensi3ve topic, don’t want to provide informa3on… Factors affec3ng the response rate: o Dura3on: we should minimize the effort to answer or offer something to par3cipate o Respondent’s percep3on of the effort required to answer o Interest in the topic o Characteris3cs of the sample (can be out of the sampling frame) o Credibility of the organiza3on responsible for the study o Induced mo3va3on level o Confiden3ality Measures to reduce the non-response: o Incen3ve o Follow-up leEer/message o Return envelope (RSF)  o Panels: ask if they want to answer again. If yes, they go to survey panel (they provide feedback) o Telephone/mail pre-no3fica3on o Closed box (closed ques3ons) o Leave collect ques3onnaire (quicker) A#en&on: have panel could be good since it reduces &me and effort to get a sample. However, it could bias our results because we are just studying one type of person. So, we should adopt a panel together with new interviewees to have a be#er/complete sample (a good approach is to use 80% or 40%). o Quan3ta3ve Observa3on - Record the behavioral paEerns of people, objects and events systema3cally - When we define the behaviors to be observed and the techniques to use for registra3on, we are reducing the poten3al bias by the observer - When we use this method, we should inform the interviewers about it but not say when or where we will apply it - The observer does not ques3on/communicate with the people being observed (except in Mystery Shopper) Can be… Disguised: respondents are unaware that they are under observa3on (e.g.: hidden cameras, mystery shopper). The goal is to collect facts, rather than percep3ons. Mystery Shopper: - Personal visits: How long were you in the queue? Did the person who received you apologize for the delay? - Telephone calls: How many rings were there before the phone was answered? Were you asked a password? Undisguised: respondents are aware that they are being observed (they were told, or it is obvious that someone is recording their behavior). By knowing, people tend to change their behavior and increase their ethical considera3ons (disadvantages). Mode of administra3on o Personal: the observer does not control/manipulate the phenomenon being observed. E.g.: Register the number of shoppers who enter in a store and observe the path that they follow. o Electronic: Electronic devices (rather than human observers) record the phenomenon being observed (they are con3nuously recording). E.g.: Automa3c coun3ng and barcode on products. Note: there are some types of data that we just can collect if we observe. For example, if the popula&on is made by kids and we want to evaluate a specific toy. There are some cases when is very difficult to observe. Advantages of Quan3ta3ve Observa3on o Measure the actual behavior rather than intended/preferred behavior o Reduce/eliminate the poten3al bias caused by the interviewer o Certain types of data can only be collected by observa3on (e.g.: children and their toy preferences- they can’t talk) o When the observed phenomenon occurs frequently, or it is of short dura3on (cost less and is faster than survey techniques) Disadvantages of Quan3ta3ve Observa3on o Difficulty in understanding the reasons for certain behavior (e.g.: purchase can be made for another person) o Bias (observer and observed person) o Data can be 3me consuming and expensive to collect (when the observed phenomenon doesn’t occur frequently, or it is of long dura3on) o Private behaviors (can’t be observed) o Ethical considera3ons (hidden cameras can violate people’s privacy) Organiza3on of the Survey Fieldwork o Moments for conduc3ng interviews: - Regular data collec3on - Con3nuous data collec3on - Data collec3on in specific moments o Professionals must: - Select the right respondents - Mo3vate them to par3cipate in the study - Promote true answers - Correctly register the answers - Conduct (report) the results for analysis Fieldwork I. Selec3on: - Consider the nature of the study and the method of data collec3on - Characteris3cs the fieldworkers should have: Communica3on, appearance, educa3on and experience (some3mes having experience is not good since they normally have “addic3ons” that can lead to bad behaviors II. Training: - Ini3al contact/presenta3on - Ask the ques3ons correctly (not change order and the way they’re wriEen) - Guide the interviewee - Register the answers - Finish the interview - Importance of the interviewer in data collec3on (honesty, objec3vity, professionalism) - Confiden3ality (of the respondent and of the client) - Familiarity with specific terminology  - Importance of following the ques3onnaire with precision (exact wording and exact instruc3ons) - Guiding the respondent without bias - Explain well the instruc3ons III. Supervision (3 types of control): - Quality control during the ques3onnaire applica3on - Sampling control (correct selec3on of respondents) - Central control (fulfillment of quotas (metas- can be 50% women and 50% men for example), key ques3ons, maximum propor3on of non-responses per interview) Note: CATI telephone interview- supervision at a distance can be be#er in the beginning but during all interviews can lead to discomfort. We may also have supervision side by side. IV. Valida3on: - Quality control ater the interview (re-inquiry) - To check quality, we should interview fewer individuals (the % depends on the size of the sample that was already interviewed), because 50% from 100 is not the same as 50% from 2000, and the 3me to re-inquiry will be very different. - If an individual doesn’t remember having done the telephone interview but the interviewer says he did, it could happen that a different person answered the phone (e.g.: father and son with the same name) V. Evalua3on: - Cost and 3me - Response rates - Quality of interviewing - Quality of data - (these are also in Supervision!) Quotas – we should check several 3mes along the study if we can accomplish them or not. o If a company wants to measure employee sa3sfac3on, it should adopt an anonymous approach. For example, using the internet is easier but can be difficult to achieve the ‘anonymity’ (using computer server of the company, which keeps the data), so we can use a paper interview with ‘envelopes’ method and mix every answer to guarantee that is fully anonymous. o If a company wants to do a survey to understand what the client’s needs of a specific profile are (top managers), a good op3on would be a ques3onnaire through internet or email (quicker). We should understand the characteris3cs of the popula3on because some could not trust/not have access to the internet and prefer by phone or presen3al, but others don’t have 3me unless it would be online.  o If a supermarket aims to measure the sa3sfac3on and loyalty of its customers and compare with the compe3tors, it could adopt different approaches: by email and by post which could be difficult to achieve customers from the compe3tors (company do not have data about them), or personal interviews outside the supermarkets, asking randomly (ask to compe3tor’s clients also). 4. Sampling Sample vs. Census o Sample is very different from Census o To perform Census, we need to have a large budget, a lot of available 3me and (ideally), a small popula3on. o Sample requires a smaller budget and less 3me/costs Sampling Steps 1. Define the target popula3on 2. Determine the sampling frame 3. Select the sampling method 4. Calculate the sample size 5. Select the sample 6. Collect the data 7. Control of data quality 8. Es3ma3on and produc3on of informa3on for decision support Sampling methods o Probabilis3c/Random Sampling: the probability of selec3on of each popula3on unit is known o Non-probabilis3c/Non-random/Empirical Sampling: the probability of selec3on of each popula3on unit is not known Random vs. Empirical Sampling: In random sampling, we can quan3fy the sampling error as we know the proper sizing of the sample and it contributes to reducing the bias resul3ng from subjec3vity of the interviewer’s work. To use this sampling method, we need to have a sampling frame and consequently its results depend on the quality of the sampling frame. Addi3onally, we need and can prevent and treat the non-responses. In empirical sampling, we need less 3me and costs for data collec3on. It is not required a sampling frame and the non-responses are ignored. However, there are always bias that we can’t quan3fy, and the accuracy of the results is not known. Non-probabilis3c/Empirical Sampling: o Useful when probabilis3c methods are impossible to implement, by not having an available sampling frame. o Useful for the exploratory research. o Useful for the pilot test of a ques3onnaire. Types of Empirical Sampling o Convenience Sampling: uses the individuals that are available, therefore, the most convenient ones. The selec3on of sampling units depends essen3ally on the interviewer so it can lead to bias, due to the lack of diversity and representa3veness of the popula3on in the sample. It is the least expensive and 3me consuming of all sampling techniques. It is very useful in exploratory research for genera3ng ideas, insights and hypothesis since we need a fast reac3on to a par3cular concept. It can also be used in the pilot test of a ques3onnaire, but we should always be careful when interpreta3ng the results o Judgmental Sampling: the units are selected on the judgement of the researcher, since he chooses the ones he believes are representa3ve of the target popula3on. Clearly, this leads to bias. This method requires low cost and fast implementa3on. Nonetheless, the quality of the sample depends en3rely on the researcher’s choices. It is useful in the exploratory phase, where there is no concern with the explora3on of the target popula3on, and we want quick reac3ons based on small samples. It is of oten used in business-to-business studies, where the target popula3on is small. o Quota Sampling: the popula3on is divided into subpopula3ons, usually using sociodemographic criteria that are known to the target popula3on (genders, age group, socio-economic group, level of educa3on). Respondents (usually) are interviewed in propor3on to the size of the sub-popula3on. In this case, the quotas guarantee that the composi3on of the sample is the same as the target popula3on with respect to the characteris3c of interest. We define the composi3on of the sample but not the units to be observed. Within each quota the units are selected based in convenience or judgment sampling (that can have interviewer’s bias), so this method should be combined with others. It Is a kind of propor3onal stra3fied sampling without sampling frame (divide popula3on in segments and apply a method to each segment to guarantee that they have diversity). o I3neraries Sampling: we select a sample of households (building, stores) where we conduct the interviews, ensuring some geographic dispersion. The interviewers are usually divided into the regions and the interviewer has a star3ng point and a specific route to follow, including the shape of the path and how to choose the households for the sample. By defining this guidelines, we are trying to reduce the bias. o Snowball Sampling: an ini3al group of respondents is selected, by a probabilis3c or, more usually, a non-probabilis3c sampling. Ater being interviewed, these group of respondents are asked to iden3fy others who also belong to the target popula3on. The main goal is to es3mate characteris3cs that are rare in the wider popula3on. This approach is associated with huge bias (depends on the interviewees opinions), so we should avoid it. It is useful for exploratory research when studying a popula3on difficult to find. o Place of Consump3on Sampling: typically held at a place of consump3on (shopping center, restaurant). It is normally used for mobile or unusual popula3ons. In this  approach we must define the places to conduct the interviews (should diversify), the possible access loca3ons, the hours to conduct the interviews and a sampling of individuals vs sampling of visits, where we analyze the data considering the frequency of visit of each interviewee. Probabilis3c/Random Sampling: o Simple Random Sampling o Stra3fied Random Sampling o Clusters Sampling o Two-stage Sampling What is the best sampling method? To select the best method to adopt we should take into account the type of research, if it is exploratory or conclusive. In the conclusive we aim to es3mate the parameters (propor3on, mean and total), so it is recommended the adop3on of a probabilis3c sampling. Nonetheless, we should consider the cost, simplicity and precision of each method, according to the resources. It is possible to combine empirical methods with probabilis3c methods (sampling with several stages). A bigger sample size requires more costs to the study. However, it also contribute to increase the efficiency of the es3mators, the precision and validity of the results. How to calculate a sample size? o Need to select the parameter a priori o The variable chosen to calculate the sample size: the es3ma3on of the most important variable of the ques3onnaire, the es3ma3on of a group of variables of the ques3onnaire or the pessimis3c case (pq = 0.25) o To choose the right formula, we need to know the parameter and the sampling method we want to adopt. Several parameters to be es3mated: o Es3ma3on of a Propor3on - Formula depends on the adopted sampling method - E.g.: Simple Random Sampling Without Replacement - We need: d (absolute precision), confidence level (1-alfa), N (popula3on size) and an es3mate or approxima3on to the popula3on propor3on p) - How can we calculate an es3mate or an approxima3on to the popula3on propor3on p? It can be es3mated/approximated from previous work or obtained from the knowledge of other variables correlated with the variable under study. Also, we can do a pilot test to obtain an es3mate for p. When we do not have any ini3al es3mate for p, we may assume the pessimis3c hypothesis (pq = 0.25) o Es3ma3on of a Mean - Formula depends on the adopted sampling method  - E.g.: Simple Random Sampling Without Replacement - Needed: d, confidence level, N and an estimate or an approximation to the corrected variance of the population S^2 - How can we calculate an es3mate or an approxima3on to the corrected standard devia3on of the popula3on (S)? It can be estimated/approximated from previous work or obtained from the knowledge of other variables correlated with the variable under study. Also, we can do a pilot test to obtain an estimate for S. We can “guess” based on the response scale of the variable, by dividing the scale range by 6. If the scale is from 1 to 5, it has a range of 4, so we use the estimate for the standard deviation = 4/6 = 0.67. 5. Non-sampling Errors 2 Types of Error o Sampling Errors: arise from the observa3on of only a part of the popula3on o Non-sampling Errors: result from the incorrect/incomplete observa3ons, failures in the work of interviewers, incorrect data analysis, among others). They don’t decrease with the increase of the sampling size. It is very difficult to es3mate them, and they introduce bias in the results o Total Error = Sampling Errors + Non-sampling Errors 4 Main Types of Errors Coverage Errors: result from the difference between the target popula3on and the sampling frame (it should have all the popula3on) from which the sample is selected. Can be: - Overcoverage: sampling frame contains elements that are not part of the target popula3on - Undercoverage: some elements of the target popula3on are not listed in the sampling frame, so they can’t be selected to the sample - Duplica3on: one or more element(s) of the target popula3on has more than one occurrence in the sampling frame - Can include here the errors in the defini3on of the target popula3on (the difference between the relevant popula3on for the problem and the target popula3on defined by the researcher) - We can avoid having individuals in our sample that do not belong to our target popula3on (but were listed in the sampling frame) by using qualifica3on ques3ons Some examples: o A company wants to know the opinion of its customers about the launch of a new product (using the customer loyalty card as iden3fier): can have overcoverage errors if an individual has the card but doesn’t use it,  undercoverage errors if it only uses customers with loyalty card (company’s can have more customers) and duplica3on errors in the case that the same individual has more than one card o Supermarket wants to interview its clients. There is a random selec3on using the telephone list corresponding to its area of influence (previously defined): can have overcoverage errors by being in the area but purchase in other supermarket, undercoverage errors if they don’t have an individual’s telephone and duplica3on errors if the same individual has more than one telephone number o Mobile telecommunica3ons company has a list of all telephone numbers from its clients. There is a random selec3on using the mobile telephone numbers list: no coverage errors Measurement Errors: result from the inaccuracy of the answers that are registered. Can be caused by: - Interviewees: wrong answers by the respondents by not understanding the ques3on, not reading the ques3on properly, invasion of privacy, poor memory, answering by mistake (explain in a different way, using examples and a simple and easy layout) - Interviewers: wrong selec3on of the poten3al respondent (not following the sampling plan), error in the formula3on of the ques3ons like not reading the whole ques3on, influence on the answers, error in the register, fraud… - Solu3ons: to avoid this we need to have a good fieldwork by doing “fake” interviews simula3ng the reality, explaining all the rules, training the interviewers and inform about supervision like direct observa3on (for quality control of data), and re-inquiry - Ques3onnaire: formula3on and order of the ques3ons, ques3onnaire structure, and dura3on - Type of interview: inappropriate type of interview (e.g.: if it is a telephone interview, we can’t show images or use extensive lists of possible answers) A good fieldwork is essen&al to ensure the quality of data! Improve the work of the interviewer o Training: correct procedures for the interviews, manual of the interviewer, and simula3on of the reality o Distribu3on of the fieldwork: the sample and different segments are allocated to different interviewers. We should prevent that any interviewer works mostly in specific groups o Quality control (supervision): direct observa3on, re-inquiry the iden3fied interviewers that don’t fulfill the procedures or falsify data, periodic monitoring sta3s3cs showing the errors rates, no response of  each interview, and interviewers who are below certain limits should receive training or must be removed from the study (extreme case) Non-response Errors: result from the existence of individuals selected for the sample who do not answer to the en3re ques3onnaire (total non-response) or to some ques3ons of the ques3onnaire (par3al non-response) - Total non-response: can be consequence of unavailability, if the individual is not at home or the phone is busy (we can try in a different hour/day), refuses to answer (invasion of privacy) or don’t know/can’t answer - Par3al non-response: can be consequence of ques3ons incorrectly formulated, sensi3ve ques3ons (some people might refuse to answer) or don’t know/can’t answer Some solu3ons for non-response errors: o New contact aEempts (in different hours/days) o Guarantee anonymity of the answers (especially when we have sensi3ve ques3ons) o Iden3fica3on of the interviewers (qualified professionals) o Contact available for clarifica3on in case of any doubt o Highlight the importance of the answer and involve the poten3al respondent with the study (explain the objec3ves) o Incen3ves (vouchers, study results…) o Decrease the size of the ques3onnaire (consequently the dura3on) o Change the ques3onnaire structure (formula3on and order of the ques3ons- sensi3ve ques3ons ate the end) o Good training of the interviewers o Build a panel of respondents Panel is a list of usual respondents. Use 90% of the individuals in the panel is easier but not good since having the same sample across all research leads us to always study the same group but not the popula&on (it is important to have new elements) Non-response introduces bias in the es3ma3on. They depend on 2 factors: o Non-response rate: - Depends on several factors: target popula3on, size/structure of the ques3onnaire, data collec3on method, percep3on of the effort that the respondent needs to answer, induced mo3va3on level. - What is worst, having a non-response rate of 10% or 20%? It depends on the quality of the answers since we can have a 20% and the popula3on is well represented or 10% of just a specific segment of the popula3on (not well represented) o Difference in behavior between respondents and non-respondents We should always try to prevent this errors. However, it is o\en necessary to treat them… Methods of treatment for non-response:  o Imputa3on: each missing data is replaced by a given expected data according to other available informa3on about the same individual or similar individuals. We have mean imputa3on, random imputa3on, sequen3al imputa3on from all or specific individuals. Processing Errors: data incorrectly transcribed from the ques3onnaires (on paper) to an electronic format. Can result from - Wrong coding - Inappropriate use of data analysis techniques - How to prevent them? Use CATI, CAPI, CAWI, edi3ng programs and use quality control in all ac3vi3es related to data processing Total Error: - The validity of the results is a func3on of the minimiza3ons of these errors: sampling and non-sampling errors - They can progress in opposite direc3ons - Depending on the available budget and the expecta3ons regarding the rela3ve weight of these errors, we may need to make a trade-off - It is not always good to have a bigger sample since it can reduce sampling errors but increase non-sampling errors (balance and try to avoid both) 6. Presenta*on of Results How can we make a good report? o The report should be wriEen for the specific readers (usually marke3ng decision-makers or managers) o Technical jargon (specialized language of a field that other may find difficult to understand) should be avoided o Report should meet the objec3ves o Appearance of report is important (paper quality, typing, colors…) 4 basic criteria to consider in the report prepara3on: o Complete: provides the reader all the necessary informa3on in a language he understands, and it should be neither too short nor too long o Accurate: some imprecision sources are simple errors in sums/subtrac3ons (sum of percentages is not equal to 100%), confusion between percentages and percentage points, inaccuracy caused by gramma3cal errors, and wrong conclusions caused by confusing terminology o Easy to follow: good organiza3on of the report and order the data analysis by theme/sec3ons o Concise: select what is included in the report and don’t include topics that aren’t directly related to the study The report may be organized in different ways, but a typical structure is: 1. Title page: 3tle of the report, name of the researchers/organiza3on conduc3ng the study, name of the client for whom the report was prepared, date 2. Table of Contents: list of the topics covered and the respec3ve page number. Includes chapters and sec3ons (all numbered) and may include a list of the tables or graphs 3. Execu3ve Summary: it is a very important part of the report, and should resume its main topics, namely a concise descrip3on of the problem, research design, major results, conclusions and recommenda3ons. Usually without sec3ons and it is the last part of the report to be wriEen 4. Introduc3on: presents the informa3on needed for the reader to understand the main text of the report. Contains specific terminology, the context of the study, the goals of the study and the organiza3on of the report 5. Body: here we should avoid excessive technical terminology. Includes: - Methodology: reasons for choosing the methodology followed, explana3on of the sampling plan and the data analysis techniques used - Results: detailed results supported by tables and figures. Summarizes the most detailed and complex results that should be presented in aEachment, and it is organized in a logical order. Each graphic should have a goal, presents sugges3ve/informa3ve 3tles, name the axes and maintain the same scale (for variables that can be directly compared), set the scale and highlight the measurement units, use sub3tles (inside the graph) when needed, present a table of data whenever it is required a more detailed analysis of the results, and assign universal connota3on to each color (e.g.: red is nega3ve, green is posi3ve, use different colors for each brand) - Limita3ons: relevant limita3ons, accuracy of the results, imprecision sources (namely non-sampling errors) and explain how the results can be generalized 6. Conclusions and Recommenda3ons: different conclusions are presented arising from the different results, with more detail than in the execu3ve summary. There must be a conclusion for each goal or problem of the study 7. Appendices: includes all the material that is too complex, detailed, specific or is not absolutely necessary for the text. It might present a copy of the data collec3on tools, a detailed calcula3on suppor3ng the sample dimensions, sta3s3cs tests, and tables not included in the main text 8. References We should be careful with the scale we use! Types of graphs o Typical goal: - Magnitude: iden3fy the importance of aEributes and show performance in aEributes - Bar chart and Radar chart - Growth, trends and changes: show progress in sa3sfac3ons and monitor evolu3on in specific aEributes - Line chart and Bar chart - Composi3on: show the composi3on of a database of clients - Round (or pie) chart and Stacked Bar chart