Chapter 5.pdf

Stuvia - Koop en Verkoop de Beste Samenvattingen Chapter 5: Innovate by rapid experimentation To (digital) Innovation = any change to a business product, service or process that adds value. - This change can range from an incremental improvement to the creation of something totally new and unpreced...

Stuvia - Koop en Verkoop de Beste Samenvattingen Chapter 5: Innovate by rapid experimentation To (digital) Innovation = any change to a business product, service or process that adds value. - This change can range from an incremental improvement to the creation of something totally new and unprecedented. - For example, For Google, an innovation may be launching a completely new product such as Gmail, Google Maps, or its Chromebook laptop line. - But innovation at Google also includes the continuous process of refining, adding and subtracting features, and evolving the user interface and experience. From Decisions made based on intuition and seniority Testing ideas is expensive, slow, and difficult Failure is avoided at all cost Focus is on the ‘’finished’’ product Decisions made based on testing and validating Testing ideas is cheap, fast, and easy Failures are learned from, early and cheaply Focus is on minimum viable prototypes and iteration after launch The fourth domain of digital transformation is innovation—the process by which new ideas are developed, tested, and brought to the market by businesses Experimentation is learning - The goal of a business experiment is not a product or solution; it is learning about customers, markets, and possible options that will lead you to the right solution. - Don’t try to avoid wrong ideas; rather, aim to test as many quickly and cheaply promising ideas as possible to learn which ones will work. - Don’t have internally debates about ideas before picking a solution and testing it. Put ideas, in rough form, in front of those who have to use the final product. - Experimenting is an iterative process of learning what does and does not work Paradigm shift: from innovation based on analysis and expertise to innovation based on ideation and experimentation for constant learning. Gedownload door: matsmolenberg | [email protected] Dit document is auteursrechtelijk beschermd, het verspreiden van dit document is strafbaar. ¤ 912 per jaar extra verdienen? Stuvia - Koop en Verkoop de Beste Samenvattingen Two types of experiments 1. Convergent experiments (Formal) = Best suited for learning that eliminates options and converges on a specific answer to a clearly defined question 2. Divergent experiments (Informal) = best suited for learning that explores options, generates insights, asks multiple questions at the same time and when done right: generates new questions to explore in the iterative stage. Convergent experiments - Seeks to provide an answer (confirmatory) - Needs a representative customer sample (test and control groups) - Needs a statistically valid sample - Focused on direct causality - Goal is to test the thing itself - Useful for optimization - Common in late stages of an innovation Divergent experiments - May provide an answer or raise more questions (exploratory) - Needs the right customers (who might not be average customers) - Sample size may vary - Focused on gestalt effects and meaning - Goal is to test as rough a prototype as possible (“good enough?”) - Useful for idea generation Example of convergent experiments: A/B testing, where two sets of customers see the same webpage for instance, with one difference in design and the company measures any difference in customer behavior. Convergent experimentation measures causality. Divergent experimentation: are generally not built around a causal question. - Rather than looking for customer response in terms of numbers, they are looking for qualitative feedback - Common mistakes in divergent testing mostly center on testing too late, as when “product testing” of a new innovation occurs after development is nearly complete You need both experimentations: To innovate successfully, you will need both convergent and divergent experiments at different stages and in different parts of your business. Successful innovation must balance both exploratory learning (to generate and develop new ideas) and confirmatory learning (to verify and refine ideas). Gedownload door: matsmolenberg | [email protected] Dit document is auteursrechtelijk beschermd, het verspreiden van dit document is strafbaar. ¤ 912 per jaar extra verdienen? Stuvia - Koop en Verkoop de Beste Samenvattingen Why digital is impacting both Convergent experimentations is becoming increasingly powerful and affordable due to new technologies. Divergent experimentation is gaining new tools from digital technology, particularly in the form of new ways to prototype ideas cheaply and quickly to customers. Seven principles of experimentation To create the most value for your innovation efforts, a few principles are critical. These seven principles apply for any business experiment, whether convergent or divergent: 1. Learn early - Starting from the very beginning lets you learn as early as possible in the process. - “The value of early learning” (or “the cost of late learning”) means that it will cost more money the later you come to realise that the customer does not want the product. 2. Be fast and iterate - The second key principle of experimentation is speed - The primary goals as a leader is to get his teams to learn faster—in iterative cycles of days rather than weeks or months – as this can become a competitive advantage. - Increasing the speed of experimentation may require infrastructure, too 3. Fall in love with the problem, not the solution - This keeps you focused on the customer and their needs => Customer value - Focusing on the problem you to consider more than one possible solution. - If your goal is the solution itself, you stop generating new ideas. - You become attached emotionally to a creative solution. If you think you only have one idea, you are unwilling to let it go. Thus, you continue driving up the costs of that one idea, trying to make it work. Therefore, don’t focus on the Gedownload door: matsmolenberg | [email protected] Dit document is auteursrechtelijk beschermd, het verspreiden van dit document is strafbaar. ¤ 912 per jaar extra verdienen? Stuvia - Koop en Verkoop de Beste Samenvattingen solution, but on the problem. 4. Get credible feedback - Credibility starts with the people you speak to: they need to be real (potential) customers, not colleagues who give feedback. 5. Measure what matters now - As interactions become more digitized, the number of things that can be measured is growing, and it is easy to get distracted by all the numbers you could be tracking. - Solution: try to identify the most important single metric for the success of your innovation (= “the one metric that matters”) 6. Test your assumptions - It is essential to test assumptions to eliminate risk, especially for innovations that take your business into unknown territory. - If your idea is based on assumptions about customers, their interests, and their willingness to pay for such an innovation, test your assumptions in a series of experiments! 7. Fail smart - Failure is inevitable. We can define failure as trying something that doesn’t work - If you try to avoid any failures, you will retreat into whatever seems most sage and you will never innovate. - But when you fail, you should fail smart. We can think of smart failure as one that passes these four tests: 1. Did you learn from the failed test? 2. Did you apply that learning to change your strategy? 3. Did you fail as early and cheap as possible? 4. Did you share your learning (so that others in your organization won’t make the same mistake)? Tool: the convergent experimental method This experimental method is particularly useful for innovating on existing products, services, and processes; for optimizing and continually improving them; and for comparing versions in the later stages of an innovation process Step 1: Define the question and its variables In a convergent experiment, the question must be as specific as possible and should, if possible, be framed as a causal question: If we do X, then what will happen to Y? - Once you have stated the question, you need to translate it into an independent variable (X) and a dependent variable (Y). Step 2: Pick your testers (who will conduct the experiment) - Gedownload door: matsmolenberg | [email protected] Dit document is auteursrechtelijk beschermd, het verspreiden van dit document is strafbaar. ¤ 912 per jaar extra verdienen? Stuvia - Koop en Verkoop de Beste Samenvattingen - Because it follows experimental practices, the test will require some statistical knowledge or tools. Many tests can be automated via software tools. Your employees can easily be trained to run and record these kinds of experiments. Step 3: Randomize your test and control - Randomly assign members of the population you want to test to a test group and a control group. - Most mistakes in convergent experiments happen here! Businesses often carefully pick who will go into the test group and who will go into the control group. • E.g., 30 highest performing stores vs 30 lowest performing stores = bad. Step 4: Validate your sample - First identify your unit of analysis: if testing an offer sent to individuals, then the unit of analysis is the individual respondent. Then, your sample size is the number of units that you place in each of your test and control groups. Typical rule of thumb: sample size of n = 100 as minimum in each group you compare Step 5: Test and analyse - Your testing team has to analyse the data to see whether there are (statistically significant) differences in the dependent variables you are measuring. Step 6: Decide - Make a decision based on the findings. If you find desired results, you should even go for further iteration and testing of additional ideas to see if they can lead to even greater improvement. Step 7: Share the learning - Critical to share your findings to others in your organisation who could benefit from this (and could avoid making the same mistakes). Gedownload door: matsmolenberg | [email protected] Dit document is auteursrechtelijk beschermd, het verspreiden van dit document is strafbaar. ¤ 912 per jaar extra verdienen? Stuvia - Koop en Verkoop de Beste Samenvattingen Tool: the divergent experimental method - - Particularly useful for innovations that are less defined from the outsets, such as new products, services, and business processes for your organisation. Divergent innovation projects tend to be very iterative and may span weeks/months. Step 1 – Step 3 = preparation phase Step 4 – Step 8 = iteration phase Step 9 – Step 10 = Action Four paths to scaling up an innovation There are four general paths for scaling up an innovation to a full release. To understand which path you should take, you need to answer two questions: 1. Can you iterate this offering quickly after launch? 2. Can you limit your rollout to stages, or does the innovation have to be released to all customers at once? Your answers to these two questions will place you in one of four quadrants. Let’s look at the requirements for successfully scaling up an innovation in each quadrant: 1. MVP Roll-Out - This is the easiest path for introducing an innovation because you can start your rollout with a limited test market and then iterate rapidly as you gain additional feedback from customers. - Your first public release will be a minimum viable product offered to a limited set of customers. - You can iterate and learn with real customers with much public scrutiny, 2. MVP launch Gedownload door: matsmolenberg | [email protected] Dit document is auteursrechtelijk beschermd, het verspreiden van dit document is strafbaar. ¤ 912 per jaar extra verdienen? Stuvia - Koop en Verkoop de Beste Samenvattingen - The second path for scaling up is harder More difficult because your business is forced to iterate very quickly after launching your innovation because you are not able to effectively limit the scope of the launch. One reason this path may be necessary is because the business has to rely on network effects. 3. Polished rollout - Harder than the first because, even though you are able to launch your innovation in limited locations or for limited customers, you cannot quickly iterate it once it is public. It therefore needs to be much more polished at the point of release. - But you can take advantage of rolling your innovation out in stages by validating your initial findings and testing how it is received by different customers or in different markets. 4. Polished launch The third path for scaling up is also harder than the first—but for different reasons. - You can launch your innovation in limited locations or for limited customers, but you cannot quickly iterate it once it is public. It therefore needs to be much more polished at the point of release This creates maximum pressure for your company to polish and carefully test an innovation before its public release. This path of innovation belongs to new automobiles, pharmaceuticals, and hardware products. Problems occur when trying to release an unpolished innovation. E.g., Google Glass was released publicly while it was still buggy and before Google was even clear on the value proposition for the customer. Organisational challenges of innovation Four organizational challenges: putting rapid experimentation at the heart of innovation is hard for many large or traditional firms. 1. Building a test-and-learn culture: companies can compensate for the fallibility of management’s own judgment if they instil in their employees a culture of testing and learning about every aspect of their business. 2. Leading without deciding: Some leaders envision great things which they immediately implement without testing their assumptions. This can cause great disasters to occur when eventually their assumptions don’t turn out to be the way they expected them. 3. Involving everyone: You can either engage the entire organisation via innovation “boot camps” or via training everyone in the organisation to adopt experimental methods in their daily work. The latter being the hardest approach. Gedownload door: matsmolenberg | [email protected] Dit document is auteursrechtelijk beschermd, het verspreiden van dit document is strafbaar. ¤ 912 per jaar extra verdienen? Stuvia - Koop en Verkoop de Beste Samenvattingen 4. Planning to fail and celebrating it: Hardest challenge for organisations is accepting, planning for, and even celebrating failure. Learning through failures is the process that takes the firm to the goal of great innovation. - Incremental innovation efforts: biggest risk is risk aversion. - Loss of learning: when failures are punished, there is no incentive to bring failures to the light. This reduces learning from them. - Throwing good money after bad: when failures are punished, any team with a budget will find a way to justify their underperforming initiative as “just needing a little more time”, adjusting their future projections and endlessly postponing any decision to shut it down = “zombie projects” To avoid these 3 hazards, businesses need to plan to fail and celebrate smart failure. Gedownload door: matsmolenberg | [email protected] Dit document is auteursrechtelijk beschermd, het verspreiden van dit document is strafbaar. ¤ 912 per jaar extra verdienen?