Conducting Experiments for Understanding Customer Preference PDF

Conducting Experiments for Understanding customer preference Experimental Design 1 Experimental Design 2 3 OR  Experimental Design Independent Dependent Variable Variable Control...

Conducting Experiments for Understanding customer preference Experimental Design 1 Experimental Design 2 3 OR  Experimental Design Independent Dependent Variable Variable Control Manipulation Variable  Experimental Design Variable Example OR Independent Variable Color Dependent Variable Sales, Attitude, Intention Control Shape, Size, Customer Characteristics Manipulation Blue vs Green  Why Experiments?? Testing new ideas Business Decisions Customer Insights Cost Benefit Analysis  Types of Experiments Experiments with Increased Behavioral realism Field Experiments Natural Experiments Methodological Consideration in Experiments Sample Size and Sample Selection Between, within or mixed Manipulation Check Attention Check Randomization Validity  Type of Experimental Design Between Subject Design Within Subject Design Participants are exposed to only one of the Participants are exposed to all conditions. conditions (treatment or control). Easier to setup Shorter Sample Size Simple Data Analysis Minimize the random noise Shorter Session Cost Effective  Manipulation Check Questions to check the effectiveness of intervention 4 4 Source is trustworthy  Attention Check Questions to check whether participants are paying attention or not Asking about experiment (Indicate the product) Asking one question several times (Name, DOB) Response Time Reverse Scaling  Barriers in Experiment Randomization  Measurement  Return on Experimentation Organizational inertia  Steps in Experiments Brainstorm Ideas Select best Ideas (Pilot Studies) Decide the Variables Hypothesis Run the experiment Analyze the results  Experiments Example 1. Product Website or App (Design, layout) User Experience 2. Price Pricing Plan (Monthly, Yearly, Daily) Pricing Plan Highlight Offers Price and Quality (High and Low, $10, $20, $30) 3. Place In store Experience (Layout, Music, Fragrance) New Channel of Distriubution  Experiments Example 4. Promotion Email Design (Headline, Copy) Advertising (Visual, Content) Social Media Ads (Call to Action) Influencer Platform Personalization Brand Name  5 Tax Experiments Original Version Modified Version Impact We are writing to inform you “By now, 9 out of 10 people in that we have still not your town have paid their received your tax payment of taxes.” £___. It is imperative that you contact us. [Contact “You are currently in the very information here] HMRC small minority of people who 35.8% --- 37.8%. have not paid us yet”  Tax Experiments Groups Independent Variable Dependent Variable Control Group: People Language of the tax Payment rates who received the letter Treatment group: original letter 37.8% Treatment Group: Control group: People who received the 35.8% modified letter  5 Microsoft Advertisement Experiment Original Groups Version Modified Version Impact Million of Dollars  5 Alibaba Cart Experiment Discount or No Discount??  5 Book My show Experiment Upfront or Backend?? Data?? Criteria?? Book My show Experiment  Probability of a purchase,  Amount spent per purchase,  Probability a customer would return for future purchases  T- test What is t test? A t-test is a statistical test that is used to compare the means of two groups. It is commonly used to determine whether there is a significant difference between the means of two groups, or to determine whether the mean of a single group is significantly different from a hypothesized value. Type of t test  Independent sample t test: To compare the mean of two groups)  Paired sample t test: compare the means of two groups that are related to each other (such as before and after measurements for the same group).  Single sample t test : To compare the mean of one sample with some predefined value Hypothesis: Null hypothesis: There is no significant difference between the means of the groups being compared. Alternative hypothesis: There is a significant difference between the means. Test Statistic If the t-statistic is high and the probability is low, then you can reject the null hypothesis and conclude that there is a significant difference between the means of the groups.  T- test One tailed or two tailed?? One-tailed test (also called a directional test) is a statistical test that is designed to test for a difference in one direction (e.g., whether the mean of one group is greater than the mean of another group). Two-tailed test (also called a non-directional test) is a statistical test that is designed to test for a difference in either direction (e.g., whether the means of two groups are different, regardless of which group has the higher mean). How to select Specific hypothesis about the direction of the difference (e.g., "Group A will have a higher mean than Group B"), use a one-tailed test. No specific hypothesis about the direction (e.g., "The means of Group A and Group B will be different"), then you would use a two-tailed test. T test decision tree T-tests Whether the observation from the same group Yes No Comparison of Comparing mean Paired Sample t- mean between against a test two groups predefined value Two Independent One-sample T-test sample T test  Single Sample t test To compare the mean of one sample with some predefined value 1. Class grade is significantly different from S No Satisfaction Score the average value 2. Life of a product is more or less than 1 80 some predefined value 2 92 3. Customer satisfaction is more than the average value 3 97 4 73 5 94 6 84 7 94 8 94 9 94  Two Independent Sample t test To compare the mean of one sample with some predefined value 1. Class grade of Section A is different form section B Satisfaction Score (Male) Satisfaction Score (Female) 2. Life of a product X is more than product Y 81 80 3. Customer satisfaction more for Male as 90 92 compared to Female 96 97 76 73 78 94 76 84 87 94 88 94 90 94  Paired Sample T test To compare the mean of same sample in two different situation 1. Class grade of Section A is different after the introduction of new teacher. Satisfaction Score (Before) Satisfaction Score (After) 2. Life of a product X is more after introduction of new technology 81 80 3. Customer satisfaction is more after the 90 92 improving the service 96 97 76 73 78 94 76 84 87 94 88 94 90 94  Multivariate Test A multivariate test (MVT) in A/B testing is an advanced experimentation method that tests multiple variables simultaneously to determine which combination of these variables performs best. Headline Variant 1: "Buy Now and Save 20%“ Variant 2: "Limited Time Offer: 20% Off“ Image Variant 1: Product Image A Variant 2: Product Image B  ANOVA What is ANOVA? ANOVA (Analysis of Variance) is a statistical method used to compare the means of two or more groups. It is used to determine whether there are significant differences between the means of different groups and can be used to test for differences between groups on a single factor or on multiple factors. Hypothesis Null hypothesis: No significant difference between the means of the groups being compared. Alternate hypothesis: Significant difference between at least two of the means. How? ANOVA takes into account the variance within each group as well as between the groups. Test Statistic F-statistic, is used to determine the probability that the null hypothesis is true. If the F-statistic is high and the probability is low, then we can reject the null hypothesis and conclude that there is a significant difference between the means of the groups.  ANOVA and Marketing 1. Testing the effectiveness of a new advertising campaign: ANOVA can be used to compare the attitude towards brand in different campaigns. 2. Comparing customer satisfaction among different demographics: ANOVA can be used to compare the mean satisfaction ratings of customers belonging to different demographic groups (e.g., age, gender, income level). 3. ANOVA can be used to compare the different price points, to determine which pricing strategy is most effective. 4. ANOVA can be used to compare different packaging designs, to determine which design is most appealing to consumers. 5. ANOVA can be used to compare the mean sales of a product at different stores that have different layout designs, to determine which layout is most effective at driving sales.  One way ANOVA Suppose a company has introduced a new flavor of breakfast cereal and wants to determine whether there is a significant difference in consumers’ purchase intention regarding boxes sold at different price points. The company has collected data on regarding different price points: $2.00 per box, $2.50 per box, and $3.00 per box. The company showed the product to the customers and asked the following question. Please indicate your intention to purchase the product on the scale of 1- 7 where 1- less likely and 7- Most likely.  Two Way ANOVA A retail chain wants to test the impact of two variables. First variable is “in store marketing”. Second variable is “coupon value”. As a marketing manager how would you design and analyze the experiment?  Experimental Design A startup which manufactures fresh juice would like to conduct marketing research regarding the purchase behavior of consumers. Past studies suggest that the two most important factors determining the preferences are price and packaging. What experiment and what analysis would you conduct to determine the effects of these factors on preference for juices?  Conceptual Model Language (Hindi vs English) Credibility of Online Reviews Modality Covariates (Text vs Audio vs Video) Covariates Age, Gender, Language Proficiency (Native, English), Product Involvement, Website Trust 17-09-2024 34  Stimuli Development Stimuli Generation 1 2 3 4 5 Platform and Fictitious Fictitious Conversion of Pretest Product Selection Webpage Review Reviews 30 Description Title Language (Back Comprehension Contact Rating: 5/1 Star Translation) Amenities Source: User 222 (Hindi/Tamil) Location Length: 150 words Modality 31 Review Language: English (Audio/Video) 17-09-2024 35  Methodology: Participants Experimental Design Sampling Participants Randomization  Snowball  Native speakers  Java Script (Hindi/English) Program  English Proficient  Experience in browsing hotel online reviews 17-09-2024 36 Essay 2 Procedure Experiment Process 17-09-2024 37 Essay 2 Results: ANCOVA Study 1 df F p η² Language 1 14.16 *** 0.06 Modality 2 12.06 *** 0.10 Language × Modality 2 0.44 0.65