2024 CMA Lecture 3_StudentsAfter PDF

Week 2 Lecture 3 Customer Marketing Analytics Measurement and Scaling: Reliability, Validity, Dimensionality Jiska Eelen Associate Professor Marketing https://research.vu.nl/en/persons/jiska-eelen  Past week: Assignment 1 Basic Statistics for Marketing § Proof of completion § Prereq...

Week 2 Lecture 3 Customer Marketing Analytics Measurement and Scaling: Reliability, Validity, Dimensionality Jiska Eelen Associate Professor Marketing https://research.vu.nl/en/persons/jiska-eelen  Past week: Assignment 1 Basic Statistics for Marketing § Proof of completion § Prerequisite to pass Customer Marketing Analytics!! § Grading will be finished before the end of the week with incomplete/complete § Deadline past Friday § Not completed yet? >> Request exemption asap. See Communication (Canvas) to contact Jiska AND Martijn § CMA = active learning; Next assignments build on prior knowledge. Don’t delay your study progress! 2 2  Today’s Lecture § Measurement and marketing § Different types of scales in marketing - Comparative scales - Non-comparative scales § Multiple item measurement theory § Scale evaluation - Reliability - Validity § Factor analysis and dimensionality of constructs (scale analysis) 3 3  How to measure? Easy to measure? Why (not)? PRICE INCOME SENSITIVITY BRAND AGE ATTITUDE CUSTOMER LOYALTY GENDER NEED FOR PARENT UNIQUENESS  LEFT HAND RIGHT HAND Easy to Difficult to observe/ask/ observe/ask/ measure measure PRICE INCOME SENSITIVITY BRAND AGE ATTITUDE CUSTOMER LOYALTY GENDER NEED FOR PARENT UNIQUENESS Simple vs. more complex variables PRICE INCOME SENSITIVITY BRAND AGE ATTITUDE CUSTOMER LOYALTY GENDER NEED FOR PARENT UNIQUENESS  Measurement and Marketing Susceptibility to Involvement Need for uniqueness social comparisons Customer loyalty Materialism Impulsive buying tendency Price sensitivity Tendency to give socially desirable Attitude answers Satisfaction § Many theories involve unobserved or latent constructs. § To test theories we need to measure these latent constructs.  Definitions § Measurement: Assigning numbers to characteristics (attributes) of objects or people according to a pre-specified rule. § Scaling: Generation of a continuum upon which measured objects are located. Example: attitude toward a brand 1 = unfavorable attitude 2 = neutral attitude 3 = favorable attitude  Measurement Scales Ratio Highest level of Scale measurement Interval Scale Ordinal Scale Metric measurement Lowest level of Nominal scales measurement Scale Nonmetric measurement scales  Measurement Scales Ratio Highest level of Net promotor score Scale measurement Interval Scale Ordinal Scale Metric measurement Lowest level of Nominal scales measurement Scale Nonmetric measurement scales  Measurement Scales Ratio Highest level of Cleanliness Schiphol Scale measurement Interval Scale Ordinal Scale Metric measurement Lowest level of Nominal scales measurement Scale Nonmetric measurement scales  Why is it important? § Identification of the measurement scale depends on the basic nature of the attribute. Measurement Scales § Critical in determining which data Ratio Scale Highest level of measurement analysis techniques are the most Interval Scale applicable to the data. Ordinal Scale Metric measurement Nominal - Chi square -> Nominal Lowest level of measurement Scale scales - Pearson correlation coefficient -> Nonmetric measurement scales Interval & Ratio - Linear regression -> DV is interval & ratio - Logistic regression -> DV is nominal  Today’s Lecture § Measurement and marketing § Different types of scales in marketing - Comparative scales - Non-comparative scales § Multiple item measurement theory § Scale evaluation - Reliability - Validity § Factor analysis and dimensionality of constructs (scale analysis) 14 14  Most Popular Scaling Techniques 1. Paired Comparison Scaling 2. Rank Order Scaling Comparative Scaling Techniques 3. Constant Sum Scale 4. Likert Scale Non-Comparative (Summated Ratings Scale) Scaling Techniques 5. Semantic Differential 15  Comparative Scaling Comparison of stimulus objects on a scale. § Paired comparison scaling Respondent needs to select one object (out of 2) according to some criterion. – Heineken or Grolsch – Heineken or Bavaria § Rank order Respondents need to order several objects according to some criterion. – 1. Grolsch, 2. Heineken, 3. Bavaria § Constant sum scaling Respondents allocate a fixed amount of points among several objects according to some criterion – Grolsch 50, Heineken 25, Bavaria 25 16  Non-Comparative Scaling Each stimulus object is scaled independently of the other object. § Likert Scale Respondents need to indicate degree of agreement to a series of statements – How much do you like the taste of Heineken? Extremely dislike 1 2 3 4 5 Extremely like - How satisfied are you with the packaging of Heineken? Very dissatisfied 1 2 3 4 5 Very satisfied 17  Non-Comparative Scaling Each stimulus object is scaled independently of the other object. § Semantic Differential Scale Rating scale with endpoints associated with bipolar labels – Please rate your perception of Heineken along the following dimensions: unpleasant __:__:__:__:__:__:__ pleasant boring __:__:__:__:__:__:__ exciting outdated __:__:__:__:__:__:__ modern inexpensive __:__:__:__:__:__:__ expensive  Non-Comparative Scaling Item Decisions § The number of scale categories: 5 to 9–point scales; depends on respondents’ knowledge and involvement § Odd vs. even number of categories – neutral response allowed or not § Nature and degree of verbal descriptions – Words, numbers, pictures – Label all vs. only extremes  Non-Comparative Scaling § Continuous Rating Scale (Slider Scale) Respondents rate the objects by placing a mark at the appropriate position on a continuous line.  Today’s Lecture § Measurement and marketing § Different types of scales in marketing - Comparative scales - Non-comparative scales § Multiple item measurement theory § Scale evaluation - Reliability - Validity § Factor analysis and dimensionality of constructs (scale analysis) 22 22  Multiple Item Measurement § Why do personality quizzes, exams and intelligence tests consist of several questions? § Why do we often use multi-item (question) scales instead of single-item scales? – To minimize error à reliability. – To capture the complex / heterogeneous nature of the construct / characteristic à validity. § Sometimes single-item scales are sufficient – Objective, concrete constructs (age, income, purchase intention) 23  Multiple Item Measurement When I go shopping, I buy things that I had A not intended to purchase. I am a person who makes unplanned purchases. B When I see something that really interests me, Impulsive Buying I buy it without considering the consequences. C Tendency It is fun to buy spontaneously. D I buy things that are not on my shopping list. E Latent variable (construct) Indicator (items) A+ B +C + D + E IBT_score = 5 Impulsive buying tendency = construct (true score) ? 24 IBT_score = operationalization (measured/observed score)  Multiple Item Measurement Xo = Xt + Xs + Xe Xo = observed score (e.g., your IBT-score) Xt = true score (e.g., your real impulsive buying tendency) Xs = systematic error (non-random error) (e.g., social desirability bias) Xe = random error (e.g., being distracted for a moment, mood) 25  Today’s Lecture § Measurement and marketing § Different types of scales in marketing - Comparative scales - Non-comparative scales § Multiple item measurement theory § Scale evaluation - Reliability - Validity § Factor analysis and dimensionality of constructs (scale analysis) 26 26  Reliability Xo = Xt + Xs + Xe Is it stable? The extent to which the measurement instrument gives the same results when the measurement is repeated. Consistency in measurement. Xe = 0. 27  Validity Xo = Xt + Xs + Xe Does it measure what it’s supposed to? The extent to which the measurement instrument indeed measures the construct that we intend to measure. With a valid measurement, differences in scores among respondents are only due to real differences in the construct (characteristic) among respondents. Xo = Xt ; Xs = 0; Xe = 0. Valid scales are reliable (reliability is necessary but not a sufficient condition for validity). Reliability and Validity Not reliable à not valid Highly reliable, but not valid Reliable and valid  Assessing Scale Reliability The extent to which the measurement instrument gives the same results when the measurement is repeated. § Two methods of assessing reliability: - Test/Re-test (Stability) - Internal consistency (Equivalence) - Split-half reliability - Inter-item (Cronbach’s alpha)  Assessing Scale Reliability Internal Consistency Method (Cronbach’s α) § Degree of agreement among scale items § Depends on number of items and mean correlation among items* § Items tapping the domain of a construct should be correlated, and if some item is not highly correlated with the others, it is probably drawn from a different domain and its inclusion produces error and unreliability. § 0 ≤ α ≤ 1. 31  Assessing Scale Reliability When I go shopping, I buy things that I had A not intended to purchase. I am a person who makes unplanned purchases. B When I see something that really interests me, Impulsive Buying I buy it without considering the consequences. C Tendency It is fun to buy spontaneously. D I buy things that are not on my shopping list. E I would rather not buy things that I did not intend to. F My partner does not like me making unplanned purchases. G  Cronbach’s α Threshold for Cronbach’s α § Fundamental research: α >.70 is acceptable, α >.80 is good. § Practical application: α >.90. Ø Improve α by deleting items with low correlation with total (sum of other items) and other items Assumption: scale is unidimensional § First use factor analysis to check dimensionality of a scale. § Then compute Cronbach’s α for each dimension. 34  Today’s Lecture § Measurement and marketing § Different types of scales in marketing - Comparative scales - Non-comparative scales § Multiple item measurement theory § Scale evaluation - Reliability - Validity § Factor analysis and dimensionality of constructs (scale analysis) 35 35 Assessing Scale Validity  Assessing Scale Validity Content Validity “Face validity” of the construct: § Do the items cover all aspects of the complex construct? (= “sampled full domain”) § Procedure - Define the domain of the construct (examine literature) - Generate many items to cover every aspect of the construct - Have experts judge whether the items fit the construct 37  Assessing Scale Validity Criterion Validity Whether a scale performs as expected in relation to other variables. § A scale is valid when it predicts well. § The correlation between scale & criterion should be high. § Example: Does Impulsive Buying Tendency predict impulsive behavior?  Assessing Scale Validity Construct Validity Does the scale measure what it should measure according to theory? § Convergent validity: Does the scale correlate positively with other measures of the same construct? - e.g., Does IBT Scale correlate highly with reported purchase frequency of typical impulse products? § Discriminant validity: Does the scale not correlate with measures of similar but different constructs? - e.g., Does the IBT Scale have a low correlation with similar but unrelated constructs such as variety seeking tendency? 39  Assessing Scale Validity Construct Validity Does the scale measure what it should measure according to theory? § Nomological validity: Does the scale behave in accordance with theory? - Create a network of relationships (=nomological net) and test whether the hypotheses are confirmed. - Eg., IBT predicts choice of hedonic vs. utilitarian products. 40  EXERCISE: Examples of testing reliability and validity (which kinds?) Measuring Consumer Innovativeness (Goldsmith & Hofacker, 1991)  EXERCISE: Examples of testing reliability and validity (which kinds?) Measuring Consumer Innovativeness (Goldsmith & Hofacker, 1991) 1. The item with the lowest item to total correlation was deleted if deletion increased alpha. After the item was deleted, the process was repeated until a final scale was determined with an alpha coefficient of.80 (Churchill 1979, p. 68), and deletion of the next item would decrease alpha. 2. The questionnaire was pretested with 27 students to ensure understandability of all the items. Unclear items were removed. 3. To improve the xxxx of the scale, experts were asked to indicate which aspects of innovativeness were not yet represented in the scale. 4. The scale items were preceded by a variable that we judged to reflect the behavior of interest in this study. Respondents were exposed to a list of four new rock record titles and were asked to indicate whether they were aware of the title and had purchased it (Yes, No, Don't Know). The titles were solicited from sales clerks at local music stores. The total of Yes responses (out of 8) served as the variable of interest. We found a positive correlation between the Innovativess variable and Rock Music Expertise, indicating the xxxx of the scale. 5. As expected, the scale correlated with consumers’ overall trait innovativeness. 6. The scale however did not correlate with need for uniqueness, again testifying of the xxxxx of the scale. The item with the lowest item to total correlation was deleted if deletion increased alpha. After the item was deleted, the process was repeated until a final scale was determined with an alpha coefficient of.80 (Churchill 1979, p. 68), and deletion of the next item would decrease alpha. LEFT HAND TWO HANDS RIGHT HAND Reliability Don’t know Validity The questionnaire was pretested with 27 students to ensure understandability of all the items. Unclear items were removed. LEFT HAND TWO HANDS RIGHT HAND Reliability Don’t know Validity To improve the xxxx of the scale, experts were asked to indicate which aspects of innovativeness were not yet represented in the scale. LEFT HAND TWO HANDS RIGHT HAND Reliability Don’t know Validity To improve the xxxx of the scale, experts were asked to indicate which aspects of innovativeness were not yet represented in the scale. LEFT HAND TWO HANDS RIGHT HAND Content Criterion Construct The scale items were preceded by a variable that we judged to reflect the behavior of interest in this study. Respondents were exposed to a list of four new rock record titles and were asked to indicate whether they were aware of the title and had purchased it (Yes, No, Don't Know). The titles were solicited from sales clerks at local music stores. The total of Yes responses (out of 8) served as the variable of interest. We found a positive correlation between the Innovativess variable and Rock Music Expertise, indicating the xxxx of the scale. LEFT HAND TWO HANDS RIGHT HAND Reliability Don’t know Validity The scale items were preceded by a variable that we judged to reflect the behavior of interest in this study. Respondents were exposed to a list of four new rock record titles and were asked to indicate whether they were aware of the title and had purchased it (Yes, No, Don't Know). The titles were solicited from sales clerks at local music stores. The total of Yes responses (out of 8) served as the variable of interest. We found a positive correlation between the Innovativess variable and Rock Music Expertise, indicating the xxxx of the scale. LEFT HAND TWO HANDS RIGHT HAND Content Criterion Construct As expected, the scale correlated with consumers’ overall trait innovativeness. LEFT HAND TWO HANDS RIGHT HAND Reliability Don’t know Validity As expected, the scale correlated with consumers’ overall trait innovativeness. LEFT HAND TWO HANDS RIGHT HAND Content Criterion Construct The scale however did not correlate with need for uniqueness, again testifying of the xxxxx of the scale. LEFT HAND TWO HANDS RIGHT HAND Reliability Don’t know Validity The scale however did not correlate with need for uniqueness, again testifying of the xxxxx of the scale. LEFT HAND TWO HANDS RIGHT HAND Content Criterion Construct  Today’s Lecture § Measurement and marketing § Different types of scales in marketing - Comparative scales - Non-comparative scales § Multiple item measurement theory § Scale evaluation - Reliability - Validity § Factor analysis and dimensionality of constructs (scale analysis) 53 53  Today’s Lecture § Measurement and marketing § Different types of scales in marketing - Comparative scales - Non-comparative scales § Multiple item measurement theory § Scale evaluation - Reliability - Validity § Factor analysis and dimensionality of constructs (scale analysis) 55 55  Factor Analysis § A multivariate statistical technique, which reduces a large number variables to a smaller number of underlying dimensions, known as factors. Ø Try to explain the maximum amount of variance (information) in the data by means of a small number of factors Ø Minimize loss of information 56  Intuition Behind Factor Analysis § How? Group together variables that are highly correlated. § Variables in the same group = high correlation § Variables in different groups = low correlation F1 =λ11 X1 +λ21 X2 F2 =λ32 X3 +λ42 X4 57  Intuition Behind Factor Analysis § Assumes interval or ratio scaled data (Pearson correlation coefficient) § Finds specific patterns in the correlation matrix (finds groups of variables with strong correlations amongst each other) § Summarizes groups of variables with high inter-correlations in terms of common factors (creates a dimension for each group of highly correlated variables) Correlation Matrix (Sorted) x3 x8 x9 x2 x6 x7 x4 x1 x5 x3 1 x8 0,73 1 x9 0,77 0,71 1 x2 0,74 0,72 0,79 1 x6 0,42 0,31 0,43 0,45 1 x7 0,47 0,44 0,47 0,49 0,72 1 x4 0,43 0,43 0,48 0,5 0,71 0,72 1 x1 0,3 0,24 0,43 0,43 0,28 0,35 0,47 1 x5 0,31 0,24 0,41 0,41 0,33 0,38 0,47 0,77 1  Intuition Behind Factor Analysis § Explain as much variance in observed variables X by means of only a few latent factors F (3 factors in the example) X3 u3 X8 u8 Not all variance F1 explained by the X9 u9 factors: u = unique X2 variance not explained u2 by the factor model X6 u6 (there is a unique term u for each item) F2: X7 u7 X4 u4 X1 u1 F3 X5 u5  Goals of Factor Analysis Key objective: Capture as much of the variance (information) in the original variables as possible with as few factors as possible § Data summarization: - Detecting structure in the relationship between variables (assessing dimensionality of constructs) § Data reduction (to a more manageable set): - Simplification for communication - Simplification for further analyses (e.g. summated scales or factor scores) 60  Too Much Data § Data summarization: - Detecting structure in the relationship between variables (assessing dimensionality of constructs) § What are the main dimensions of SERVQUAL? How can we summarize 22 items defining the domain of service quality? (Parasuraman et al. 1988)  Too Much Data § Data reduction (to a more manageable set) / simplification: Example: Understand perceptions of various brands of motorcycles amongst riders in Italy. A sample of 335 individuals rates eight brands on eight image attributes: Brands Attributes How can we reduce the Aprilia Exclusive attributes to a set of ‘new BMW Innovative variables’ (factors) and Ducati Reliable determine the position of each Guzzi Performing Harley Trendy brand on each factor? Honda Aggressive Suzuki Engaging Yamaha Winning  Factor Analysis Design § How many observations? Ø Rule of thumb: Minimum observations total = 50 Ø Rule of thumb: Minimum observations per variable 5:1 § What type of variables can be used? Ø You need a correlation measure to measure association between variables, hence metric data. § How many variables per factor? Ø Rule of thumb: if purpose of finding a structure, want ≥ 3-5 variables per factor 63  Factor Analysis Step by Step 1. Inspect correlation matrix: is FA appropriate? 2. Choose method of extraction. (Principle Component Analysis). 3. Determine number of factors underlying the data. (A-priori determination, Eigenvalue > 1, Scree Test). 4. Rotate the initial solution for proper interpretation. (Varimax). 5. Interpret the rotated solution and name the factors. 6. Calculate Cronbach a for each factor found. 7. Create summated scales or factor scores for further analysis. 64  Factor Analysis Step by Step § Survey of employees of a company (N=114) § 7-point Likert Scale (1=strongly disagree, 7=strongly agree) § 11 Questions: 1. I show confidence in my staff 2. I let my staff know they are doing well 3. I give feedback to staff on how well they are working 4. I would personally compliment staff if they did outstanding work 5. I believe in setting goals and achieving them 6. I achieve the things I want to get done in a day 7. I never try to put off until tomorrow what I can finish today 8. I plan the use of my time well 9. I remain clear headed when too many demands are made upon me 10.I rarely overlook important factors when plans are made 11.I handle complex problems efficiently 65  Correlation Matrix: Is FA Appropriate? § Bartlett’s Test of Sphericity provides statistical significance that the correlation matrix has significant correlations among at least some of the variables. § H0: all possible correlations between variables = 0 § χ2 should be large, p 0.5 Ø Bartlett’s test of sphericity- H0 is rejected (p < 0.05)  How many factors? (EV>1) 3 factors capture 69% of variance that is in the original 11 variables. 3 Eigenvalues > 1 à 3 factors 71  Number of Factors: Eigenvalue >1 § Each factor has its own eigenvalue § Eigenvalue = amount of variance explained by factor § Sum of all eigenvalues = number of variables % of variance explained by a factor = Eigenvalue Factor/Number of variables § Rule for selecting factors: Eigenvalue > 1 § We want each factor to explain the variance of at least a single variable (Note: Each variable has a variance of 1) How many factors? (Scree plot) “Elbow” rule: § Look for the “elbow” (break, sudden flattening in Elbow at k = 4 eigenvalues) (e.g., k) à 3 factors § Take the number of factors before the elbow (k-1) 73  Communalities Extracted Communalities Initial Extraction I show confidence in my 1.000.593 staff I let my staff know they are 1.000.803 All communalities doing well I give feedback to staff on are very high, so no 1.000.560 how well they are working variable should be I would personally compliment staff if they 1.000.704 eliminated. did outstanding work I believe in setting goals 1.000.660 and achieving them I achieve the things I want 1.000.727 to get done in a day I never try to put off until tomorrow what I can finish 1.000.710 today I plan the use of my time 1.000.772 well I remain clear headed when too many demands 1.000.720 are made upon me I rarely overlook important factors when plans are 1.000.650 made I handle complex 1.000.676 problems efficiently Extraction Method: Principal Component Analysis.  Communalities Extracted § The factor extraction analysis computes “communalities” of items with the common factor structure. § How much of the variance of each variable is captured by the extracted factors. § If communality is very low (say <.30), the item is “quite unique” since it correlates weakly with other variables. Such variable should be removed, as it is definitely measuring “something else.” § Since communality = row sum of squared factor loading -> communalities depend on the number of factor in your solution (more factors -> higher communalities)  Interpretation of the Rotated Solution Rotated Component Matrixa Component 1 2 3 I show confidence in my -.023.721.270 staff Naming the Factors I let my staff know they are doing well.239.848.162 § Factor 1 (planning) I give feedback to staff on.156.705.195 how well they are working I would personally compliment staff if they.271.779.153 § Factor 2 (participation did outstanding work I believe in setting goals.746.312 -.079 with staff) and achieving them I achieve the things I want.793.061.307 to get done in a day I never try to put off until § Factor 3 (handling tomorrow what I can finish.787.251.167 today stress/efficiency) I plan the use of my time.817.040.322 well I remain clear headed when too many demands.072.283.797 are made upon me I rarely overlook important factors when plans are.226.375.677 made I handle complex.296.122.757 problems efficiently Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization. a. Rotation converged in 6 iterations. 76  Interpretation of the Rotated Solution § Rotated Component Matrix presents factor loadings of each item = correlation between factor and each original item § Items with high loadings (i.e., close to 1 or -1) on a given factor are used for interpreting the factor § Factor loadings can be positive or negative (depends on item scaling) à consider absolute values of factor loadings for interpretation § Look for simple structure: each variable (hopefully) loads high on 1 factor and low on other factors § When interpreting highlight the highest factor loadings per row, and then look at highlighted items for each factor  Calculate Cronbach’s a for each factor § After a factor analysis, select high loading variables on each factor and conduct reliability analysis Ø Recode items with negative factor loading Ø Calculate resulting Cronbach’s a Ø To increase reliability, look at item-total correlations and Cronbach’s a if item is deleted 78 Calculate Cronbach’s a for each factor Analyze > Scale > Reliability Analysis … Calculate Cronbach’s a for each factor § Factor 1 (planning) Cronbach’s a = 0.84 § Factor 2 (participation with staff) Cronbach’s a = 0.82 § Factor 3 (handling stress/efficiency) Cronbach’s a = 0.76 80  Create summated scales or factor scores for further analysis § Next, we can compute summated scales or factor scores for subsequent analysis. § Three factor solution suggests three summated scales (or three factor scores) should be constructed. § These three new variables (replacing the original set of variables) are used for further analysis. 81  Summated Scales: Compute Mean Scores on Dimensions Compute the mean of the high loading variables in each dimension In SPSS: Transform > Compute Variable 82 We had 11 columns. Now only 3.  Factor Scores Score of the individual respondent on the factor (incorporates all variables) In SPSS: § Analyze > Data Reduction > Factor... § Button: Scores  Conclusion § The 11 managerial skills can be grouped into 3 super-variables (dimensions): Planning, participation with staff, handling stress/efficiency § The scale is reliable: Cronbach’s a for each dimension >.70 § We can use the resulting 3 summated scales for subsequent analyses. § On average, how does Company X do on employee’s managerial skills? § How do employee’s managerial skills at Company X and Company Y compare? § To reduce high correlation among predictors in a regression. E.g., How do managerial skills predict organizational performance? To test whether impulsive buyers are more interested in using our brand, I would measure “product interest” and “impulsive buying tendency”. Next, I would test if the items of impulsive buying tendency load on one factor. Finally I would use xxxxx as the IBT score and correlate it with product interest. RIGHT HAND LEFT HAND The factor TWO HANDS The score (mean of I don’t know summated the items score (mean multiplied by of all items) their factor loading)  Video Tutorial Scale Analysis Case Tuesday, September 10th Assignment 2 Opens Discussion Board Opens 87 Thursday, September 12th Q&A Session Friday, September 13th Assignment 2 Deadline 88  Lecture 4 Tuesday, September 10th: Creating Perceptual Maps (using Factor Analysis)