Marketing Management Tutorial 4 PDF

MA ASTR ICHT UNIV ER SI TY MARKETING MANAGEMENT TUTORIAL 4 TODAY’S TARGETS 01 02 03 Understand the basics Understand the value Apply the knowledge in of data collection and of experiments in an in-class exercise (qu...

MA ASTR ICHT UNIV ER SI TY MARKETING MANAGEMENT TUTORIAL 4 TODAY’S TARGETS 01 02 03 Understand the basics Understand the value Apply the knowledge in of data collection and of experiments in an in-class exercise (quasi-)experimental business environments. methods.  01 Icebreaker TUTORIAL 02 Web lecture basics AGENDA 03 A/B tests and experiments 04 In-class activity ICEBREAKER: TRUE OR FALSE? Nike's "Just Do It" slogan was inspired by the last words of a notorious criminal before execution. The slogan was inspired by Gary Gilmore’s last words, "Let's do it." Dan Wieden of Wieden + Kennedy advertising agency adapted it to "Just Do It" for Nike's campaign. ICEBREAKER: TRUE OR FALSE? The "Like" button on Facebook was originally going to be called the "Awesome" button. Facebook developers originally considered naming the "Like" button the "Awesome" button, but ultimately decided on "Like" for its simplicity and universal appeal. ICEBREAKER: TRUE OR FALSE? Sony's PlayStation was originally intended to be a joint venture with Nintendo. Sony originally worked with Nintendo to create a CD-ROM for the Super Nintendo, but the partnership fell through, and Sony subsequently developed the PlayStation independently. ICEBREAKER: TRUE OR FALSE? The first Starbucks store opened in Seattle sold both coffee beans and musical instruments. The first Starbucks, opened in 1971 in Seattle, sold high-quality coffee beans and equipment. It did not sell musical instruments, although Seattle is known for its rich musical culture. ICEBREAKER: TRUE OR FALSE? Before settling on the name "Pepsi," the drink was originally called "Brad's Drink" after its inventor, Caleb Bradham. Pepsi was indeed originally called "Brad's Drink" when it was created in the 1890s. It was renamed "Pepsi-Cola" in 1898, inspired by the digestive enzyme pepsin and kola nuts used in the recipe.  Brand Brand Brand Logo statement name WHAT WE Build a brand identity ACHIEVED SO Create a product idea FAR AND WHAT IS NEXT Collect survey data Measure the effectiveness of brand elements Statistical analysis COVERING THE BASICS OF THE WEB LECTURE MEASUREMENT …finding some way of assigning numbers, or labels, or some other kind of well-defined descriptions, to “stuff” Measurement is the process of assigning numbers, labels, or well-defined descriptions to characteristics, attributes, or phenomena (called constructs) to quantify, categorize, or describe them systematically. The theoretical construct The measure(ment) i.e., the thing to be measured e.g., numbers, labels, etc. Age 33 years Efficacy of a new drug 50% reduction in symptoms Satisfaction of the service 7 (= very satisfied) OPERATIONALIZATION Operationalization is the process by which we take a meaningful but somewhat vague concept and turn it into a precise measurement. - A precise definition of what is measured. - Determining the method of measurement. - Choosing a level and range of measurement. How can you measure happiness? What do you think is a better measure of happiness? - Wealth/income - Number of smiles in a week LEVELS (SCALES) OF MEASUREMENT A variable is the quantifiable data collected from individuals, objects, or phenomena when a measurement method is applied to assess a theoretical construct. Not all variables are equal. Different types of variables are called levels (or scales) of measurement. Nominal Ordinal Interval Ratio LEVEL PROPERTIES True zero Equal Equal interval interval Rank order Rank order Rank order Categories Categories Categories Categories Nominal Ordinal Interval Ratio  Rating the fruit from Ranking the fruits Categorizing the fruits 1 (=no sweetness at Counting the grams of from most to least into sweet and sour. all) to 10 (=extremely sugar in the fruits. sweet. sweet). Nominal Ordinal Interval Ratio EXERCISE (I) Colors Levels of Years of pH levels preferred Loyalty experience Brand Number of Brand names Temperature rankings bottles sold Nominal Ordinal Interval Ratio QUASI-INTERVAL SCALES: LIKERT SCALES A Likert scale is a type of survey question that lets you express how strongly you agree or disagree with something using a numbered range, like 1 for 'strongly disagree' to 5 for 'strongly agree’. On my next shopping occasion, I would consider buying [BRAND NAME]: Ordinal Interval TRANSFORMING THE LEVELS Data can be transformed from one level to another. Transformations are unidirectional: Ratio → Interval → Ordinal → Nominal Transforming data results in a loss of information. Rating the fruit from Ranking the fruits Categorizing the fruits 1 (=no sweetness at Counting the grams of from most to least into sweet and sour. all) to 10 (=extremely sugar in the fruits. sweet. sweet). Nominal Ordinal Interval Ratio CONTINUOUS AND DISCRETE VARIABLES Discrete variables: A discrete variable is one that can only take on distinct, separate values, typically counts of items or occurrences. For example, the number of students in class. Continuous variables: A continuous variable is one that can take on any value within a given range, representing measurements that can be infinitely divided. For example, the time students prepare for class. Always discrete Discrete or continuous Nominal Ordinal Interval Ratio ASSIGNING A ROLE TO VARIABLES We often aim to understand some variables in terms of some other variables. - Outcome variables (dependent variables): to be explained - Predictor variables (independent variables): to do the explaining Is an asymmetric (vs. symmetric) logo preferred for exciting brands? Predictor variable Outcome variable Asymmetric vs. symmetric logo Please rate how much you like the logo from 1 (=strongly dislike) to 5 (=strongly like) MEASURING THE EFFECTIVENESS OF BRAND ELEMENTS 1) Understand the relationship that is being tested → brand elements (predictor) impact brand identity and preference (outcome). 2) Define your theoretical constructs → What is brand identity and preference? 3) Operationalize your measurement → How to measure identity and preference? What levels of measurement do you choose? What are valid measures of identity and preference? 4) Create a survey that follows your operationalization. 5) Collect data from your desired sample. 6) Choose the correct statistical tests for analysis. TESTS NEEDED FOR BRP (AND EXAM) Level of Measurement Used in Statistical test Predictor Outcome BRP Report One-sample t-test Continuous Brand Statement Independent samples t-test Continuous Brand Logo Paired samples t-test Categorical Continuous ANOVA Continuous Brand Statement Chi-square test of independence Categorical Brand name BRP GUIDELINES FOR BRAND ELEMENTS Brand Statement Brand Logo Brand names 2 1 3 Participants are randomly How many? Participants see one brand Participants see all brand assigned to one brand logo statement to evaluate names side-by-side to evaluate Preference, Liking, Outcome Brand Personality Choice Attitude, etc. Geuens et al. (2011) A valid measure chosen Operationalization Side-by-side choice brand personality scale by you One-sample t-test Independent-samples t- Chi-square test of Test ANOVA test (“A/B” testing) independence  RELIABILITY AND VALIDITY Reliability refers to the degree to Validity refers to the extent to which a measurement or which a tool or method accurately assessment tool produces stable measures the construct it is and consistent results over time. intended to assess. If you use your kitchen scale to If the kitchen scale shows the weigh a bag of oranges several correct weight according to a times, and it gives you the same standard measure (like another reading every time, that scale is trusted scale), then it's valid reliable. because it's giving you the true weight of the bag of oranges. VALIDITY Validity refers to the extent to which a tool or method accurately measures the construct it is intended to assess. Internal validity refers to the extent to which you are able draw the correct conclusions about the causal relationships between variable. vs External/ecological validity refers to the extent to which your results generalize to settings beyond your study. Construct validity refers to whether you’re measuring what you want to be measuring. AN EXAMPLE OF CONSTRUCT VALIDITY Imagine a marketing team that wants to measure the effectiveness of their online advertising campaign. They decide to use web traffic as an indicator of success, assuming that more web traffic means more people are interested in their product. What are the pros and cons of using web traffic? ACHIEVING CONSTRUCT VALIDITY Complex and difficult to formally establish → goes beyond this course Best practices: - Don’t create your own measurements unless necessary. - Most constructs have been measured in some way or form in the past. Marketing researchers measure (brand) attitude, liking, preference, willingness to buy/pay, etc. all the time. - Find high-quality peer reviewed research to look for measurements. Adapt if needed. Logo liking from Bajaj and Bond (2017) Brought to you by… WHAT IS AN A/B TEST AND HOW DOES IT RELATE TO EXPERIMENTS? A/B TESTING AND EXPERIMENTS WHAT IS AN EXPERIMENT? Experimental research involves the researcher's control over variables, with the manipulation of independent variables to observe their effects on dependent variables, using randomization to minimize biases. Non-experimental research lacks the manipulation of variables seen in experimental designs, often due to ethical or practical constraints, making it more susceptible to confounding variables Quasi-experiments resemble controlled experiments but do not manipulate independent variables, instead observing naturally occurring variations with more caution in result interpretation. A/B TESTS “A fancy way of saying independent-samples t-test” From Facebook A/B testing: Typical performance measure: Click-through rate (CTR): 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝐶𝑙𝑖𝑐𝑘𝑠 𝐶𝑇𝑅 = × 100 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝐼𝑚𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛𝑠 A/B TESTS Example from Kohavi and Thomke: Sample of 32 million users Bing ran a set of experiments with the colors of various text that appeared on its search results page, including titles, links, and captions. Darker colors lead users to be more successful in their searches, requiring less time. Color changes would increase revenue by more than $10 million annually A/B TESTING KEY BENEFITS Data-driven decisions: Replaces guesswork with hard data. Minor changes with potentially major impact: Small tweaks can significantly boost revenue. Fast iteration: Quick idea validation enables agile adaptation. Investment insight: Guides resource allocation with precise impact analysis. Evidence-based strategy: Elevates decision- making to a scientific process. A/B TESTING POTENTIAL RISKS Data Integrity: Accurate A/B testing requires data validation, like A/A tests, to eliminate distortions from anomalies such as bot traffic. User Response Variance: Significant differences in how user segments respond to A/B tests can skew results, necessitating detailed analysis. Causality Confusion: A/B tests must carefully distinguish causality from mere correlation to avoid false conclusions about the effectiveness of changes. IN-CLASS ACTIVITY: LLAMA JUICE IN-CLASS EXERCISE Imagine you are a consultant, supporting clients with insights from marketing research. Your current client is a soda brand called “Llama Juice”. Llama Juice is planning to run a marketing campaign on Instagram. The creative team came up with several ads. Based on your expertise, you need to measure which advertisement has the greatest chance of success. PART I Come up with three metrics that measure the effectiveness of an advertisement. - One metric should be continuous and on a ratio scale. - One metric should be categorical (not a Likert scale). - One metric should be using an interval scale (e.g., a Likert scale). Look for a validated scale from Bruner (2021) or come up with your own scale. Check both! For each scale, think carefully about operationalization. - What is a definition of “effectiveness”? How can this construct be measured. - How exactly does your measurement work (e.g., what options are there)? Also think about construct validity. Why do you think the measures are measuring what you are trying to measure? A/B TESTING KEY BENEFITS Next, think about how you can compare the effectiveness of two ads. How can an A/B test help you to pick a “winner”? Imagine that instead of an A/B test, you show both ads side-by-side in a survey. Participants pick their preferred ad. How does this setup differ from an A/B test? What are advantages and disadvantages?  THANK YOU AND SEE YOU NEXT SESSION! PREPARE THE MANDATORY MATERIALS EVERY GROUP MEMBER NEEDS QUALTRICS ACCESS