Marketing Management Tutorial 5 PDF

M AASTRICHT UNIVERSITY MARKETING MANAGEMENT TUTORIAL 5 TODAY’S TARGETS 01 02 03 Understand the basics Understand how to Begin creating a of sampling. create a good questionnaire to test questio...

M AASTRICHT UNIVERSITY MARKETING MANAGEMENT TUTORIAL 5 TODAY’S TARGETS 01 02 03 Understand the basics Understand how to Begin creating a of sampling. create a good questionnaire to test questionnaire. the effectiveness of your brand elements.  01 Theoretical Background TUTORIAL 02 Questionnaire best practices AGENDA 03 Questionnaire creation TEXTBOOK BASICS TWO TYPES OF STATISTICS Descriptive Statistics: Describing the features of a dataset in statistical terms. What is the mean age of the UM student body? ! Mean: 𝑥̅ = ∑"#$! 𝑥# " Inferential Statistics: Drawing inferences from a sample about a larger population. Does a newly developed COVID vaccine reduce the likelihood of getting infected? H0 vs. H1 SAMPLING Sample: Actual data that is available to researchers. A subset of the population the researcher wants to draw conclusions about. Population: All possible observations the researcher wants to draw conclusions about. HOW TO DEFINE YOUR POPULATION? SAMPLING THEORY In a simple random sample, every member of the population has an equal chance of being selected. Stratified sampling: Divide population into subgroups (strata) before random sampling. Snowball sampling: Current study participants recruit future participants. In convenience sampling, participants are selected based on their availability and ease of access to the researcher ISSUES OF CONVENIENCE SAMPLING Self-selection Non-representative sub-populations: You are WEIRD Western, Educated, Industrialized, Rich, and Democratic Students YOUR OWN CONVENIENCE SAMPLE For your group project, you are required to collect at least 100 responses. Groups on social Your network Recruitment media Crowdworker ($$$) INCENTIVES FOR STUDY PARTICIPATION/COMPLETION ‘Hard’ incentives: ‘Soft’ incentives: - Money - Being nice - Gift card lotteries - Transparency and honesty - Gifts (free coffee, cookie, …) - Good study design - Course credit - Interesting topics - Office contributions - Short surveys -… -… THE LAW OF LARGE NUMBERS Sample statistics can differ from population statistics. The larger your sample, the closer the sample mean will be to the population mean. THE CENTRAL LIMIT THEOREM (CLT) The central limit theorem states that the distribution of sample means approximates a normal distribution as the sample size becomes large. Why do we care: - Larger sample sizes are more reliable than smaller ones. - The normal distribution is the basis of many statistical tests à We are building confidence intervals (CIs) to make claims about differences. What is a large sample size? - Conventionally, a sample size of 30 or larger suffices for the CLT to hold. CONFIDENCE INTERVAL A 95% Confidence Interval is a range derived from sample data that has a 95% chance of containing the true population value under repeated sampling. QUESTIONNAIRE DESIGN FROM RESEARCH QUESTION TO QUESTIONNAIRE (I) 1) What do you want to find out? - What is your research question and hypothesis? - What are your predictor and outcome variables? 2) What data do you need to answer your research question? - Operationalize your theoretical constructs. - Choose method of measurement. 3) How do you analyze the data? - Different data structures require different skills. Be sure that you have the skills to analyze the data. 4) How can the data be collected? - Choose the right questionnaire elements to collect the required data. FROM RESEARCH QUESTION TO QUESTIONNAIRE (II) 1) What do you want to find out? - Testing the effectiveness of brand elements. - Brand elements à effectiveness 2) What data do you need to answer your research question? - Brand statement à Geuens et al. (2009) brand personality scale - Brand logo à some evaluation measure of your choosing - Brand name à choice 3) How do you analyze the data? - Brand statement à ANOVA, one sample t-test - Brand logo à independent samples t-test - Brand name (and influence of gender) à Chi-2 test of independence (DIS)ADVANTAGES OF QUESTIONNAIRES Advantages Disadvantages Cheap Non-engagement and lying Simple and quick Incomplete responses Adaptable and scalable Misunderstandings Most question types are easy to Limited way of providing analyze additional information  QUESTIONNAIRE FLOW Get to your main (research) Questions that are not part Participants start by giving questions early. Ask of your main (research) Close with demographics, consent and reading the questions that require much question or that are easier to survey feedback, and if instruction page. thought early. answer should come later. necessary, a debrief form. Partic i p a nt m o ti v ation Effort required Consent Instruction Demographic Feedback Debrief Main part form page information form form Time QUESTIONNAIRE ELEMENTS: CONSENT FORM Purpose: - Informs participants about the study's nature, procedures, risks, and benefits in a language that is easy to understand. - Documented agreement showing the participant’s voluntary participation with full knowledge of their rights and the research implications. Elements: - Title of the Study: Clearly states the research title. - Purpose: Explains the aim and objectives of the research. - Deception: At any point in this study, is the participant deliberately deceived? - Procedures: Describes what participants will do during the study. - Duration: Indicates the time commitment required from participants. - Risks: Outlines any potential risks or discomforts. - Benefits: Describes the potential benefits to participants or society. - Confidentiality: Assures participants of privacy and explains data handling. - Voluntary Participation: States that participation is voluntary and can be withdrawn. - Contact Information: Provides researcher's and IRB's contact details for questions. - Consent Acknowledgment: Includes a statement where participants acknowledge their understanding and agreement to participate, typically followed by an "I agree" option for online forms. QUESTIONNAIRE ELEMENTS: INSTRUCTION PAGE Briefly describes the objective and importance of the survey, ensuring participants understand the context and relevance of their responses. Tell participants what you want: - Gut responses or long deliberation? - Factual information or opinions? - Respond as yourself or as an average consumer? Best practices - Keep the language simple that avoids ambiguity - Outline the order of the questionnaire - Brevity is key QUESTIONNAIRE ELEMENTS: MAIN BODY This part contains your actual (research) questions. For academic research, work “backwards”: start with measuring the outcome variable before measuring the process variable. For non-academic research, get the most ”valuable” data first. 2 Subjective arousal 1 Logo Perception of symmetry excitement TWO GENERAL QUESTION TYPES Open-ended Closed-ended Open-ended questions prompt people to Closed-ended questions limit answers. answer with sentences, lists, and stories. Provide richer data Provide very specific data Help with vaguely defined theoretical constructs Allow to establish statistical relationships Allow nuance and clarifications Quickly answered Time-consuming Can be misinterpreted Hard to analyze and compare (also LLMs made this easier) Allow only limited insights AI-generated “Good” responses hard to distinguish from ”bad” responses. How satisfied or dissatisfied are you with this How satisfied or dissatisfied are you with this process? Please write down your thoughts in process? Please answer on a scale from 1 (not at all the text box below. satisfied) to 7 (very satisfied) FROM QUESTION TO DATA This part contains your actual (research) questions. Question names logo_liking become variable (column) names. 4 5 6 CLOSED-ENDED QUESTION TYPES Multiple choice questions form the basis of most research. They can be displayed as a traditional list of choices or as a dropdown menu, tick box, radio buttons, etc. Multi-select is used when you want participants to select more than one answer from aa list. Ranking order is used to determine a respondent’s order of preference for a list of items. These questions are best used when you want to measure your respondents’ attitude toward something. These can be time-consuming for respondents. Rating order questions are asked to indicate their personal levels on things such as agreement, satisfaction, or frequency. Scale questions. Rather than asking respondents a basic yes or no question, use scales that measure both the direction and the intensity of opinions. Matrix Tables are used to collect multiple pieces of information in one question. They provide an effective way to condense your survey or to group similar items into one question. Sliders let respondents indicate their level of preference with a draggable bar rather than a traditional button or checkbox. … COMMON MISTAKES Mistake Definition Example Response options overlap, not Non-Exclusive Answers "Age groups: 18-25, 25-30, 30-35" mutually exclusive. Double-Barreled Two questions combined into one. "Do you enjoy running and swimming?" Questions Suggesting a particular answer in Leading Questions "Don't you agree our product is the best?" the question. Complex Wording & Using complicated language or "How do you rate our CRM's UX/UI?" Jargon specialized terms. Questions presuppose specific "How effective is the Heisenberg Assuming Knowledge information or expertise. compensator?" "What are your thoughts on the various Long & Ambiguous Overly lengthy or unclear aspects of current environmental policies in Questions questions. relation to global economic trends?" CHATGPT IS YOUR FRIENDLY HELPER J QUESTIONNAIRE ELEMENTS: DEMOGRAPHICS Demographics are statistical data relating to the population and particular groups within it, such as age, gender, income level, education, and ethnicity. Think carefully about which demographics are needed for your (research) question and/or to describe your sample. Allow participants to skip these questions. Be aware of GDPR guidelines! QUESTIONNAIRE ELEMENTS: FEEDBACK AND DEBRIEF Allow participants to provide feedback in an open-ended manner. Here, they might mention challenges they were facing. If the research included deception (i.e., participants were deliberately misinformed about the true purpose of the study to ensure the validity of the results), provide a debrief page that states the true nature of the experiment or contact information where the information is available. QUESTIONNAIRE EXERCISE CREATE YOUR QUESTIONNAIRE YOUR QUESTIONNAIRE ELEMENTS Brand statement à Geuens et al. (2009) - Brand statement should not contain brand name or logo - Brand statement should not contain words from Geuens et al. (2009) scale Brand logo à A/B test - Find your own evaluation measure - Brand logo should not contain brand name Brand name à Side-by-side choice + other required questionnaire elements QUALITY CHECKS AND PEER FEEDBACK Test your questionnaire several times before distributing it: - All relevant elements are present - Checked for spelling and grammatical errors? - Avoided common mistakes? - Checked whether your randomization works? à are both logos shown? - Responses are requested to minimize missing data? - Checked “recode values” to see whether scale points and values are assigned correctly. Peer feedback: - When your survey is ready, send it to your peers - A à B, B à C, C à A - Please provide feedback within 24hrs!  THANK YOU AND SEE YOU NEXT SESSION! FACILITATED BY GROUP A SHARE YOUR CASE AT LEAST 48HRS BEFORE CLASS