Analyze Attribution Models PDF

Perform an analysis Introduction When performing an analysis, it’s important to use the right data, matched with the right methodology. To conduct statistical and data analyses, you need to interpret statistical outputs, consider validation metrics, create data visualizations and write a simple sc...

Perform an analysis Introduction When performing an analysis, it’s important to use the right data, matched with the right methodology. To conduct statistical and data analyses, you need to interpret statistical outputs, consider validation metrics, create data visualizations and write a simple script or query to extract or manipulate data. Identify and interpret statistical outputs. Why are these important? Statistical outputs help you to interpret the results of your analysis, while validation metrics allow you to measure the quality of the analysis. Together, they indicate how robust your data is. Consider the value of each statistical output across different media channels and platforms. Almost every output has a validation metric, so it’s important to consider both when conducting an analysis. You can distinguish the value of statistical outputs across media channels and platforms through the validation metrics. PERFORM AN ANALYSIS 33 Examples of outputs and validation metrics Review some examples of outputs and validation metrics in the graphic below. Both Statistical outputs Regression coefficients R-squared Slope and intercept Correlation coefficient (R) Mean, median, mode Standard errors (SE) Confidence Intervals Mean error Bias and variance Validation metrics Log-likelihood Standard deviation (STDEV) P-values Adjusted R-squared T-static F-static Durbin-Watson Variance inflation factor Sample size Choose an attribution model. Given the validation metrics, recommend an attribution model. Single-touch attribution models Multi-touch attribution models First click or visit Even credit Last touch Positional Data-driven attribution model Time decay PERFORM AN ANALYSIS 34 Single-touch attribution models Single-touch attribution models give credit to only one touchpoint. First click or visit What it is This model gives 100% of the credit for a conversion to the first click on the conversion path. How it’s used Use this model to better understand how to value the first click or visit in a conversion path, especially when success is defined within a longer attribution window or a longer consideration period. It gives no credit to impressions or later touchpoints that could have incremental effects on your conversion rate, and may oversimplify conversion paths that rely on middle- and lower-funnel activity. If you want to understand and credit the full conversion path, consider even-credit, positional or time-decay attribution models. How it’s calculated This model is a rules-based single-touch attribution model. It gives 100% of the credit for a conversion to the first click or visit in a conversion path. If a click and visit happen within 60 seconds of each other, then only the click is counted. Example If a conversion path contained an impression first, then a click, and then a visit, the click would receive 100% of the credit for the conversion. If a conversion path contained an impression, then a click, and then a visit 30 seconds later, the click and the visit would count as the same touchpoint and get 100% of the credit for the conversion. First click Visit PERFORM AN ANALYSIS 35 Last touch (including last click) What it is This model gives 100% of the credit for a conversion to the last click, visit, impression or view that happened in a conversion path. If there is no click or visit, then this model credits the last impression. How it’s used Use the last-touch model when you want to consider only the last touchpoints in a conversion path. For example, in a last-click attribution model, the last touch is a click. This model can help understand how to value the last touchpoint in a conversion path, especially when success is defined within a shorter attribution window or you have low-consideration conversions. This model does not give credit to earlier touchpoints that could have incremental impact, and may oversimplify conversion paths that rely on upper and middle-funnel activity like awareness and consideration. If your goal is to understand and credit the full conversion path, consider even credit, positional or time-decay attribution models. How it’s calculated This model is a rules-based single-touch attribution model. It gives 100% of the credit for a conversion to the last click or visit that happened in a conversion path. If there is no click or visit, then it will credit the last impression. If a click and a visit happen within 60 seconds of each other, only the click is credited. Example If a conversion path contained an impression first, then a click, and then a visit, the visit would receive 100% of the credit for the conversion. If the path didn’t contain a click or a visit, then the impression would receive 100% of the credit for the conversion. If a conversion path contained an impression first, then a click, and then a visit 30 seconds later, the click and the visit would count as the same touchpoint and get 100% of the credit for the conversion. Last touch Including last click PERFORM AN ANALYSIS 36 Multi-touch attribution models Multi-touch attribution models take into account more than one interaction with a given media channel. They are inclusive of other models, including even credit, positional and time decay, which are explained below. Even credit What it is This model considers the full conversion path and gives each touchpoint equal credit for a conversion, regardless of where it appeared on a conversion path or if it was an impression, click or visit. How it’s used This model helps you understand how to value the first touchpoint that introduced the product, the middle touchpoints that build consideration and the last touchpoint that helped people get to the point of conversion. Typically, even-credit models are more illustrative than actionable, since it’s unlikely that all touchpoints are equally effective. Compared to a last-touch or last-click model, even credit better reflects how all touchpoints can lead to a conversion, and can inform your business decisions. If your goal is to understand and credit the full conversion path, consider positional and time-decay attribution models. How it’s calculated This model is a rules-based multi-touch attribution model. It gives an equal credit percentage to each click, visit and impression on a conversion path. If there was one impression, three clicks and one visit on the conversion path, each one would share 20% of the credit for the conversion. Example If a click and visit happen within 60 seconds of each other, then only the click is credited. If an impression and a click appear to be associated with the same ad and happen within 24 hours of each other, then they are counted as a single touchpoint when receiving credit for a conversion. Even credit PERFORM AN ANALYSIS 37 Positional What it is This model gives a specific percentage of credit for a conversion to the first and last touchpoints in a conversion path, with the remaining credit distributed evenly across all intermediate touchpoints. This model considers the full conversion path, but gives weighted credit to the first and last touchpoints. How it’s used This model helps you understand how to value the touchpoints that occurred first and last in a conversion path. This model typically values any middle touchpoints with less credit than the first and last. Compared with a last-touch or last-click model, positional better reflects how all touchpoints helped lead to a conversion while considering the important roles that the first and last touchpoints may have played. If your goal is to understand and credit the full conversion path, you should also consider even-credit and time-decay attribution models. How it’s calculated This model is a rules-based multi-touch attribution model in which the first and last touchpoints are given a specific percentage of credit and the remaining credit is distributed evenly across all other touchpoints. The positional model is offered in two configurations, 30% and 40%, where either 30 or 40 percent of the credit is given to both the first and last touchpoints, with the remaining 40 or 20 percent of the credit distributed evenly among the remaining touchpoints Example If you choose Positional 30%, and there are five touchpoints in your conversion path, the first touchpoint will receive 30% of the credit, the last touchpoint will receive 30% of the credit and the remaining three touchpoints will each get 13% of the credit. If a click and a visit happen within 60 seconds of each other, then only the click is credited. If an impression and a click appear to be associated with the same ad and happen within 24 hours of each other, then they are counted as a single touchpoint when receiving credit for a conversion. Positional PERFORM AN ANALYSIS 38 Time decay What it is This model gives an increasing percentage of credit for a conversion to touchpoints as they get closer in time to the conversion. This model considers the full conversion path, but gives weighted credit to touchpoints as they get closer in time to a conversion. How it’s used This model helps you understand how to value the multiple touchpoints that helped lead to a conversion, but gives the most recent touchpoints more credit. Compared with a last-touch or last-click model, time decay better reflects how all touchpoints lead to a conversion in a way that may more realistically represent how customers interact with and consider ads as they get closer to converting. To understand and credit the full conversion path, consider even credit and positional attribution models. How it’s calculated This model is a rules-based multi-touch attribution model. It decreases the amount of credit given to each touchpoint by half after a set amount of time, with more credit given to the most recent touchpoints. This model is offered in two configurations, 1 day and 7 day half-life. A longer half-life leads to a more even distribution of credit over time, whereas a shorter half-life distributes a majority of credit to the most recent touchpoints. Example Choosing a one day half-life means that touchpoints that happened one day before the conversion get 50 percent of the credit, and touchpoints that happened two days before get 25 percent of the credit. Time decay PERFORM AN ANALYSIS 39 Data-driven attribution model The data-driven attribution model assigns fractional credit for a conversion to Meta touchpoints based on their estimated incremental impact. This is a statistical-based model developed by Meta and updated periodically. It has coefficients that can vary from advertiser to advertiser, industry to industry and analyst to analyst. All other models, last click, last touch, even credit, positional and time decay, are rule-based models, which follow a set of finite and transparent rules and a predetermined formula. Because this model uses learnings from actual data observations and is trained on randomized control experiments, it can more accurately measure the incremental value of your marketing efforts. This model is available on Facebook, Instagram, Audience Network and Messenger only. Conduct a statistical analysis. Based on your hypothesis, determine which statistical analyses are required: Simple linear regression Multiple linear regression Logistic regression T-Test Chi-Square Test F-Test ANOVA Correlation Given a set of coefficients, the functional form of a model and a data table, calculate the output column. See an example of how it could be calculated below. Note that this sample data is independently compiled and not the output delivered from an actual test. PERFORM AN ANALYSIS 40 Reconcile differences across different measurement solutions. Because not all measurement tools use the same methodology, you may have to reconcile results across different measurement solutions. To do that, ask yourself: ● ● ● ● ● ● ● What data input is used? Identify which events are included in the data, whether they are accurately monitored and how data is collected across devices. What time frames are used? What conversion window is applied? What attribution window is used? What measurement methodology is used? Does it use an observational method, like attribution or marketing mix modeling? Or an experimental method, like lift studies? Does it include the use of historical data or data from similar campaigns or industries? Examples of reconciling differences across measurement solutions Platforms such as YouTube or Twitter may define a video view, or thruplay, differently, so it’s important to consider what counts as a video view. Conduct a data analysis. To kick off a data analysis, make a table that breaks down summary statistics. Summary statistics give a quick and simple description of the data. They can include mean, median, mode, minimum value, maximum value, range and standard deviation. For example: The standard Ads Manager attribution window is set to 1-day view and 28-day click, but in Google Ads conversion reporting is 30 days, so attribution windows need to be reconciled before analysis. Data Data 34 56 Mean 65 112 Standard error 7.63201278 78 Median 56 65 Mode 55 44 Standard deviation 29.5586584 55 Sample variance 873.714286 75 Kurtosis -0.9432981 23 Skewness 0.18550307 110 Range 92 98 Minimum 20 97 Maximum 112 53 Sum 975 55 Count 15 20 PERFORM AN ANALYSIS 41 Calculate lift based on experiment data. Breaking down a summary statistic is the first step of most analyses. After, you can perform additional analyses, such as lift calculations. Lift results include information about the results your ads caused, including the metrics: Lift % Indicates how much your ads increased the rate at which people converted (as defined by the conversion events you chose when you created the test). For example, if your ads increased the conversion rate by 50%, that could mean that you got 100 conversions during the test without ads and 150 with them. That would mean that they got you 50 additional conversions. Lift % calculations Divide the number of additional conversions by the number of conversions you would’ve gotten without ads and multiply that by 100 to calculate Lift%. In this case, that would be: ● 50 / 100 = 0.5 ● 0.5 x 100 =50% Conversion lift The number of conversions that wouldn’t have happened without your ads. Confidence A percentage that represents how confident Meta is that your ads caused conversion lift. Results that Meta is at least 90% confident in are considered reliable. Meta’s testing methodology includes thousands of simulations based on your test. If your ads caused conversion lift in 80% of Meta’s simulations, Meta would be 80% confident that your ads caused conversion lift during the test. PERFORM AN ANALYSIS 42 Write simple script or query to extract and manipulate data. Raw data for analysis often exists in databases and other sources and needs to be retrieved. Data that exists in various databases or data tables needs to be combined and filtered to extract what is suitable to enter into an analysis. A JOIN clause combines rows from two or more tables, based on a related column between them. Inner join Returns records that have matching values in both tables. Left join Returns all records from the left table and the matched records from the right table. Right join Returns all records from the right table and the matched records from the left table Full join Returns all records when there is a match in either left or right table. Read SQL Joins for examples about how to apply the different JOINs. Inner join table 1 table 2 Left join table 1 table 2 Right join table 1 table 2 Full outer join table 1 table 2 Imagine a retail analyst has a database that contains comprehensive data for stores of all sizes from the last 5 years, but they only want to analyze data from stores in California that are larger than 500 square feet and from the last 12 months. The analyst queries a database to pull such data using SQL and then conducts some of the analysis, including visualizing the data, creating descriptive statistics and running a form of statistical analysis such as regression. An appropriate JOIN based on data structure would combine data from multiple tables. PERFORM AN ANALYSIS 43 Create data visualizations. Data visualization is a graphic representation of data. It involves producing images that communicate relationships among the represented data to viewers. This communication is achieved through the use of a systematic mapping between graphic marks and data values to create a visualization. While this is not required, it’s recommended as a good practice for analysts. Heat map Jan Feb Mar Jul Aug All Placements Feed In-stream video Explore Stories Bubble chart PERFORM AN ANALYSIS 44 Line graph Scatter plot PERFORM AN ANALYSIS 45 Bar chart PERFORM AN ANALYSIS 46