Reading_-_Case_Study_NoodleAI%20(1)%20(2).pdf

CASE: SM-301 DATE: 07/15/18 NOODLE ANALYTICS IN 2018: AI FOR THE ENTERPRISE A nagging feeling was eating at Bradley Stewart, the CEO of XOJET. suspected, was leaving money on the table.1 His company, he The Silicon Valley private jet firm had celebrated its 10-year anniversary in 2016, and things...

CASE: SM-301 DATE: 07/15/18 NOODLE ANALYTICS IN 2018: AI FOR THE ENTERPRISE A nagging feeling was eating at Bradley Stewart, the CEO of XOJET. suspected, was leaving money on the table.1 His company, he The Silicon Valley private jet firm had celebrated its 10-year anniversary in 2016, and things were looking up. Though the company had weathered some rough times after the 2008 financial crisis, annual revenue had risen from $100 million in 2009 to $400 million in 2015.2 The company had grown into the third-largest private aviation company in the world (after NetJets and Flexjet), focusing on high-volume business and luxury travelers. But something seemed to be missing. “Like a lot of other business leaders, we’re reading the Harvard Business Review, and you can’t avoid the narratives around artificial intelligence,” said Austin Schell, XOJET’s president of fleet operations. “We were asking ourselves: Are there ways that we can use data analytics specifically, or machine learning more broadly, to add value to our business?”3 Coincidentally, XOJET’s controlling shareholder, TPG, had just made an unusual investment. The private-equity giant wasn’t known for funding brand new start-ups; it had made its name buying out companies like Continental Airlines and J. Crew. But in early 2016, TPG allocated $15 million to seed a San Francisco-based AI start-up with the quirky name Noodle.ai.4 1 See XOJET video at https://www.youtube.com/watch?v=D7PUQfGm0UU (August 15, 2018). Andrew Cave, “Share My Plane: Can XOJet Join Uber and Airbnb In The Sharing Economy?” Forbes, February 15, 2015, https://www.forbes.com/sites/andrewcave/2015/02/15/share-my-jet-can-xojet-ever-join-uber-and-airbnbin-the-sharing-economy/#27863cd2e92d (June 13, 2018). 3 Interview with Austin Schell on April 23, 2018. All quotations are from this interview unless otherwise noted. 4 “Leading Experts in Artificial Intelligence Launch Noodle.ai,” TPG press release, March 14, 2016, http://press.tpg.com/phoenix.zhtml?c=254315&p=irol-newsArticle&ID=2148986 (June 13, 2018). Julie Makinen and Professor Robert A. Burgelman prepared this case as the basis for class discussion rather than to illustrate either effective or ineffective handling of an administrative situation. 2 Copyright © 2018 by the Board of Trustees of the Leland Stanford Junior University. Publicly available cases are distributed through Harvard Business Publishing at hbsp.harvard.edu and The Case Centre at thecasecentre.org; please contact them to order copies and request permission to reproduce materials. No part of this publication may be reproduced, stored in a retrieval system, used in a spreadsheet, or transmitted in any form or by any means –– electronic, mechanical, photocopying, recording, or otherwise –– without the permission of the Stanford Graduate School of Business. Every effort has been made to respect copyright and to contact copyright holders as appropriate. If you are a copyright holder and have concerns, please contact the Case Writing Office at [email protected] or write to Case Writing Office, Stanford Graduate School of Business, Knight Management Center, 655 Knight Way, Stanford University, Stanford, CA 94305-5015. This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. Noodle Analytics SM-301 p. 2 Noodle.ai’s founders, Stephen Pratt and Raj Joshi, believed that the time was ripe for AI for the enterprise, focused on predictive analytics. The two had worked together first at Deloitte Consulting and then as cofounders of Infosys Consulting (IC), a U.S.-based subsidiary of the Indian outsourcing firm Infosys Technologies. Over their seven years at IC, the company grew from zero to an $800-million-a-year global technology consulting business. Both later led the consulting and systems integration business of Infosys from 2011 to 2014. Pratt had a strong perspective on AI: “Artificial intelligence and machine learning will fundamentally change business,” he said. “Any executive team that’s not using these techniques in five years is going to be out-competed, and they’re not going to know why.”5 As a nascent company, Noodle.ai needed to figure out what kind of problems it could solve, and for what kind of clients. It needed to refine its business model and strategy; Pratt and Joshi believed they should position Noodle.ai as a product company with a Software-as-a-Service (SaaS) business model, rather than a pure consulting-driven services operation. But what, exactly, should the product be? In the summer of 2016, XOJET and the Noodle.ai team sat down and started to, well, noodle over how they might work together. They talked about predicting airport delays or the probability of a trip being disrupted before it took off. They talked about safety issues. But ultimately, they decided the real value in the business came down to better dynamic pricing, Shell recalled. XOJET had introduced dynamic pricing around 2011, but the system was rudimentary—XOJET used a 28,000-cell spreadsheet with hundreds of city pairs in it. He noted: We have a team of eight to 10 revenue managers who, historically, just based upon intuition and experience, would set the pricing level for the day. We had huge Excel spreadsheets and the pricing was pretty crude. We would price a day as either “standard,” “high,” or “low.” …It was all done via a lookup table in Excel. The problem with dynamic pricing by humans is that we’re all human, and are subject to cognitive bias. When we were ahead of [our revenue] plan, we would get greedy on price and leave assets sitting. When we were behind plan, we would get too timid and price too low and chase volume and started leaving money on the table. We met with Noodle and asked them: Could you build an algorithm … that would help us really home in on where there’s pricing opportunity? A Noodle.ai team—including a client services partner, a data scientist, a data engineer, and a user experience designer—started working side-by-side with XOJET, shadowing the employees there. “We didn’t know very much about private aviation. XOJET didn’t know very much about AI or machine learning. So there was a month or two where we were just teaching each other,” said Amit Saini, the client services partner who led the Noodle.ai team.6 Noodle.ai acquired data sets from the FAA and other public and private sources. Eventually, they came up with 3,300 different factors, from weather to holidays, which could play into a 5 6 Interview with Steve Pratt on May 9, 2018. All quotations are from this interview unless otherwise noted. Interview with Amit Saini on April 6, 2018. All quotations are from this interview unless otherwise noted. This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. Noodle Analytics SM-301 p. 3 given day’s demand. Noodle built a pricing algorithm that incorporated those factors, and a user interface that gave XOJET’s revenue managers a dashboard to easily see the optimal price to quote for a given trip. Later, they added tools to understand region-to-region demand dynamics. The results started showing up in XOJET’s bottom line: The company notched year-over-year increases in unit (i.e., per plane) revenue in 9 of the first 12 months after implementing Noodle’s machine learning model. The company also saw occupied fleet time rise by more than 900 hours, while repositioning hours (time spent moving empty planes) fell 5 percent, Schell noted. Unexpectedly, the data that Noodle wrangled also gave XOJET insights into how it was faring against competitors. For the first time, XOJET understood it had about a 35 percent market share in flights longer than 3½ hours—more than 6x its nearest competitor. But on flights between 2 and 3½ hours, it had only an 18 percent share. That inspired XOJET to create a new fixed-rate product targeted at jet card customers,7 Schell recalled. Noodle, said Stewart, “challenged our business to think about our business more critically.”8 Over the next two years, Noodle.ai would find more traction, with companies from steelmakers to tire distributors, and by summer 2018, they would even secure a new round of funding. But the initial path from XOJET forward wasn’t obvious. Among the questions facing Joshi and Pratt as they got Noodle.ai off the ground: How would the company replicate the success it found with XOJET with other types of customers? What kind of scalable, replicable products should it build to move beyond being a consultant building bespoke solutions for individual companies? And where should Noodle.ai look for clients—inside the TPG family of companies, or outside? THE ORIGINS OF NOODLE.AI At first glance, Joshi and Pratt appeared to be a bit of an odd couple. Tall and outgoing, Pratt was a visionary type who chafed at rules, thrived in unstructured environments, and could be a bit anti-establishment (though he noted having worked on satellite communications and “spooky stuff” for the government early in his career). Joshi was shorter and more soft-spoken, a details man who liked process and getting things done. He grew up in India and arrived in the United States in 1980 with just $20 in his pocket. Eventually, he earned an MBA.9 “We’re yin and yang. Raj is really good at the things I’m horrible at doing, and vice versa,” said Pratt. “I need someone who’s super detail-oriented, and loves processes… and will negotiate pricing to the last penny. That’s not me.” But both Pratt and Joshi had studied engineering, and both played tennis. Both were competitive, and had an appetite for risk. Their careers started to converge in the 1990s at 7 A jet card is a product that enables cardholders to use various private airplanes at agreed-upon fixed hourly rates. Jet cards are offered by fleet operators and charter brokers. 8 XOJET video, op. cit. 9 Interview with Raj Joshi on March 23, 2018. All quotations are from this interview unless otherwise noted. This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. Noodle Analytics SM-301 p. 4 Deloitte Consulting, where each made partner in 1995. (For more background on Pratt and Joshi’s backgrounds, see Exhibit 1.) Joshi founded Deloitte Consulting Offshore Technology Group in India and served as its CEO for more than three years. Meanwhile, Pratt had grown Deloitte’s Customer Relationship Management (CRM) practice from scratch. The two started working together to leverage the offshore model at Deloitte. In 2004, they jumped ship together to Infosys, excited about the opportunity to create a new consulting model. IC’s strategy was to deliver high-quality business consulting and disciplined technology implementation at an extremely competitive price. Whereas leading consulting firms like IBM and Accenture charged $175 to $225 per hour, IC could perform consulting engagements for a blended rate of about $100 an hour.10 IC used the general Global Delivery Model employed by Infosys Technologies. They broke projects into components and distributed the work where it could be done most cost efficiently, and used a 24-hour project workday to save time. Their consulting engagements used a framework they called the Value Realization Method, which explicitly linked their projects to customer and shareholder value with clear metrics (see p. 14 for further details). After 10 years building up IC, Pratt and Joshi left in early 2014. For a few months, they looked for a company to buy together, but nothing panned out. Joshi got several job offers and joined MicroStrategy, a Northern Virginia-based provider of business intelligence software product and visualization tools. As Pratt considered what to do next, he thought back to an AI project he had worked on for the U.S. Coast Guard in the 1990s. The project—which sought to identify ships smuggling drugs—had been a failure. He recalled: You’d put in the data, you’d hit calculate, and then you’d wait days and days and days, and it’d usually crash. If it ever got to completion, the ships were long gone. And you’d have burned through hundreds of thousands of dollars’ worth of computing power at the time. So it wasn’t possible. But I had always thought that there was something there. This fact that learning algorithms, if they could ever work, could fundamentally change business.… You could run businesses much more effectively if you could predict, and fix, problems before they happened. So that had always been in the back of my mind, that this was a very powerful tool. But by 2014, Pratt thought the world had changed. “The three things needed to do AI well— abundant data, high-performance computing, and mathematics—had finally come together.” Pratt saw AI as “power tools for the mind,” giving people newfound capacity to sense patterns they couldn’t detect before, predict things they previously couldn’t anticipate, and make recommendations based on volumes of data too large for one human to grasp. In this new era, he thought, business executives as well as political leaders would start to shift their thinking from a 10 For further information, see “Infosys Consulting in 2006: Leading the Next Generation of Business and Information Technology Consulting,” GSB No. SM-151, p. 8. This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. Noodle Analytics SM-301 p. 5 deterministic style to one that was more probabilistic, with a focus on understanding risk. He said: We’re going to have 11 billion people on the planet in 2100.… We need to get much more efficient at the use of natural resources, less waste, better decisions and allocation of resources, producing more with less. I think those are all very important things from a societal point of view, and I think artificial intelligence and machine learning hold the key to be able to process very complex situations, and make better decisions. So we have less inventory sitting in warehouses, less wasted food, better allocation of water, better allocation of energy—those kind of things. Connecting with TPG Pratt wondered if he should start an AI company. But he wasn’t sold on the idea—after all, he had essentially just spent a decade running a start-up. By chance, he connected with some TPG executives and told them about his vision for a company that would offer AI to large companies. Unbeknownst to Pratt, TPG was sensing a need for a business that would help mature companies make better sense of their data. Nehal Raj, a partner at TPG, recalled: Big companies were coming to us and saying, “We have so much data. We know the data is important. But we don’t know what to do with it, how to analyze it or where to even start. Can you help us?” It became kind of obvious to us that there was a need in the marketplace for a business that could help other enterprises make sense of their data. This is pre-Steve [Pratt], but kind of in parallel with his own journey. And what we said was, “Let’s go and find businesses that do this.” …We talked to a lot of the companies that purport to do this and found many flaws in their business models. And so, we said, “Gee, the space is really interesting. There’s clearly demand for it if you use our portfolio as a proxy for bigger enterprises. But all the companies that are out there don’t seem to be approaching it the right way, in our view.” We were just kind of stumped.11 When Pratt met with Raj, the two sides felt an immediate connection: What Pratt was talking about building was what TPG’s portfolio companies needed. “That happens like once every thousand meetings,” said Raj. “He needs an investor. We need someone with a vision.” But both Pratt and TPG thought it might be hard to build an AI company from scratch. “We immediately agreed… it was probably going to get to scale more quickly if we went out and bought something together, and have Steve and his team run it. And maybe change it, tweak it, to be what we wanted. But that way, at least we start with clients and revenue.” Pratt joined TPG as an executive in residence. He and Raj spent the better part of 2014 looking at companies to buy. “Long story short,” said Raj, “we didn’t get anything done.” They could not find a company they were enthused about. After a while, Raj said, “it started to feel like 11 Interview with Nehal Raj on May 11, 2018. All quotations are from this interview unless otherwise noted. This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. Noodle Analytics SM-301 p. 6 what we were doing was bringing Steve in to be a turn-around executive for a business that wasn’t working.” “At that point, I’d spent nine months shopping for a company,” remembered Pratt. “And I said, I can’t spend my life looking for one.” Just then, IBM called, and asked Pratt if he would be interested in running the company’s Watson12 implementations for IBM Global Business Services. IBM had created a separate business unit for Watson and its AI business at the start of 2014, and had grand ambitions for it.13 Pratt accepted. After less than a year on the job, though, Pratt found that working in a company with nearly 400,000 employees was not his speed. He got in touch with TPG again. “I said, ‘This company needs to exist.’ They said ‘Okay, well, we don’t [typically] do start-ups, but we’ll make an exception. We still have the money, and we love the investment thesis,’ and so they agreed to put in $15 million as our Series A round,” recalled Pratt. “And so then I called Raj [Joshi].” TPG and ‘De-Risking’ It took but a single conversation for Pratt to get Joshi on board. At MicroStrategy, Joshi had been hearing from clients that they wanted more help with predictive analytics. “They started identifying the need, but they didn’t know how to do it, what to do, how to get there,” said Joshi. There was a personal factor as well: While Joshi’s sons were telling him to take it easy, he was not yet 60 and had no intention of retiring; he wanted to keep doing big things. About 15 years earlier, he said, he had had an epiphany of sorts: Until then, it was… “I want to play golf. I want to drive a Porsche. I want to. I want to. I want to.” That’s what drove me. And then there was this aha moment of, “That doesn’t really matter.” …Why do we exist as human beings? We exist because we should be touching lives of others for the better. That’s what really matters. And so if you start a company and you grow the company… we are helping touch people’s lives for the better by giving them a career in the next big thing. For all their passion and good intentions, though, Pratt and Joshi were hardly AI experts. And TPG’s core business was not start-ups. Why, then, was TPG interested? Beyond Pratt and 12 Watson, IBM’s supercomputer, shot to fame after beating human contestants on Jeopardy in 2011. After that, according to the MIT Technology Review, “IBM coopted the name for a wide range of AI techniques and related applications—everything from natural language processing to medicine, voice recognition, sentiment analysis, business analytics, and more. In most cases, the roles Watson supposedly taking on involve applying some version of machine learning in a novel area.” See Will Knight, “IBM’s Watson Is Everywhere—But What Is It?” MIT Technology Review, October 27, 2016, https://www.technologyreview.com/s/602744/ibms-watson-is-everywherebut-what-is-it/ (June 13, 2018). 13 Steve Lohr, “IBM Is Counting on Its Bet on Watson, and Paying Big Money for It,” The New York Times, October 17, 2016, https://www.nytimes.com/2016/10/17/technology/ibm-is-counting-on-its-bet-on-watson-andpaying-big-money-for-it.html?_r=0 (June 13, 2018). This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. Noodle Analytics SM-301 p. 7 Joshi’s track record at IC, a big part of it was the demand for AI in TPG’s portfolio of companies. Nehal Raj recalled: We saw that there was so much demand sitting here, in our portfolio, that this wasn’t going to be your normal start-up where you create a name, you hire a few people, you build a product and you try like heck to sell it. You give it away for free to get additional customers and see what happens. That’s usually where all of the risk is in a start-up. We said we can supercharge this. The minute we start it, we’ll introduce you to 10 or 20 of our companies. …We can go and say, “Hey, you said you had this need. We started this company in response to that. Why don’t you guys talk?” The realization that we could leverage our existing portfolio to drive demand helped de-risk what otherwise would have been pretty risky— just starting a company from scratch. Pratt and Joshi may have had clients in waiting, but didn’t have a clear road map. They didn’t have any staff, or even an office. “A couple of months before launch, we built financial models, started interviewing candidates and basically put our brains into start-up mode. We knew we wanted to grow really fast but how precisely are we going to drive growth?” recalled Joshi. “ I told Steve ‘Let’s just get going and it will come.’ Having launched companies before gave us the confidence to make the leap.” The two partners started working out of Pratt’s dining room in Piedmont, California, reaching out to former colleagues and other contacts as they tried to recruit a team. Within weeks they had recruited their CFO (Anil Kumar, formerly of Infosys Consulting), CTO (Ted Gaubert, formerly of Infosys Consulting), chief data science officer (Matt Denesuk, formerly of GE) and chief HR officer (Martha McGaw, formerly of Infosys Consulting). (See Exhibit 1 for bios.) Gaubert, Denesuk and McGraw became part of the founding team on day one, while Kumar joined the company a few weeks later. They also needed a name for the company. Joshi wasn’t particularly focused on it. “I didn’t pay any attention to the name.… I said, ‘Oh, how difficult can it be to come up with a name?’ Well, it was very difficult. Every dotcom web address you can think of is taken.” Pratt and Joshi had considered hundreds of names and wanted something memorable, creative, easy to spell, and with some connection to thinking, learning, or the brain. After much brainstorming, the name “Noodle” was suggested by Chelsea Hardaway, a brand advisor (and later, Noodle.ai chief marketing officer). Noodle can mean “brain” or “to think about, as in “noodle on it” or “use you noodle.” But all that came to mind for Joshi, at first, was spaghetti. “To me,” he said, “noodle was pasta.” However, he ultimately came around: “In the end we all agreed to go with it, and now we really love our name.” As it turned out, Noodle Inc. was already taken. So was Noodle.com. But “Noodle” was memorable, so Pratt and Joshi decided to go with Noodle Analytics Inc., and registered Noodle.ai. This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. Noodle Analytics SM-301 p. 8 Pratt and Joshi incorporated Noodle Analytics in February 2016 and launched it a month later on the appropriately geeky date of March 14, 2016 — which math nerds celebrate as “Pi Day.”14 But they had much still to figure out: What would their strategy be? Who, exactly, was their competition? What kind of company culture did they want and need to create to be successful? AI INDUSTRY CONTEXT What is Artificial Intelligence? By 2016, Artificial Intelligence had moved out of the realm of sci-fi movies and well into everyday life. Technologies like Apple’s Siri and Amazon’s Alexa brought speech-recognition to mainstream consumers. Driverless cars were being tested on roads in California and elsewhere. Customer-service chatbots—computer programs designed to simulate conversations with human users—regularly interacted with humans on the web, and Google’s AlphaGo computer had bested a human world champion in the ancient strategy game of Go. But AI was more than a novelty: PwC predicted that as a result of AI, global GDP could be up to 14 percent higher in 2030 than it would be without it. Of that, $6.6 trillion was expected to come from increased productivity and $9.1 trillion from consumption-side effects.15 Like the team at XOJET, many managers at medium-to-large companies were feeling pressure to incorporate AI into their operations. An article in the November 2016 Harvard Business Review warned ominously: “Unlike with the internet, where latecomers often bested those who were first to market, the companies that get started immediately with machine intelligence could enjoy a lasting advantage.”16 Investors were rewarding companies with a strong AI focus: While Walmart had three times the revenue of Amazon in 2017, by mid-2018, Amazon (seen as an AI leader) had a market capitalization three times as great as Walmart’s.17 A study by MIT Sloan Management Review in 2017 found that 85 percent of the 3,000+ executives surveyed believed AI would allow their companies to obtain or keep a competitive advantage, while about 75 percent believed AI would allow their firms to move into new businesses.18 14 “Leading Experts in Artificial Intelligence Launch Noodle.ai,” TPG press release, March 14, 2016, http://press.tpg.com/phoenix.zhtml?c=254315&p=irol-newsArticle&ID=2148986 (June 13, 2018). 15 Anand Rao and Gerard Verweij, “Sizing the Prize: What’s the real value of AI and how can you capitalize?” PwC, January 22, 2018, p. 3. https://www.pwc.com/gx/en/issues/data-and-analytics/publications/artificialintelligence-study.html (June 13, 2018). 16 Shivon Zilis and James Chan, “The Competitive Landscape for Machine Intelligence,” Harvard Business Review, November-December 2016, p. 4. 17 Walmart’s sales in 2017 topped $500 billion, while Amazon’s revenue was $178 billion. Yet Walmart’s market capitalization stood at $248 billion on June 13, 2018, while Amazon’s was $822 billion. 18 Sam Ransbotham, David Kiron, Philipp Gerbert, and Martin Reeves, “Reshaping Business With Artificial Intelligence: Closing the Gap Between Ambition and Action,” MIT Sloan Management Review, Sept. 6, 2017. https://sloanreview.mit.edu/projects/reshaping-business-with-artificial-intelligence/#chapter-1 (June 13, 2018). This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. Noodle Analytics SM-301 p. 9 And yet, only about 20 percent said they had incorporated AI into some of their business processes or offerings, and only 5 percent had “extensively” incorporated AI. Fewer than 4 in 10 companies said they had an AI strategy in place; not surprisingly, the largest companies (those with 100,000 or more workers) were most likely to have an AI strategy. But even among the largest companies, only about half had articulated an AI strategy. Another 2017 survey of more 3,000 CIOs and other top IT leaders found that only 11 percent rated themselves as “competent” or “very competent” in their understanding of AI.19 Many people were still struggling to understand just what AI, in fact, was. The Oxford Dictionary defined AI as “the theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decisionmaking, and translation between languages.”20 But as the MIT Sloan/BCG study noted, “AI is evolving rapidly, as is the understanding and definition of the term.”21 AI could be considered a tree with multiple branches. (For a glossary of AI-related terms, see Exhibit 2.) Functional overview of various types of AI technologies22 AI had the potential to help businesses a) generate new revenue by increasing sales of existing products and services, and/or creating new products or services b) reduce costs of producing and delivering products and services, and c) improve customer service or decrease cost of customer service. But a 2018 Gartner report noted: “Enterprises are struggling to identify where and how 19 Erick Brethenoux, “Artificial Intelligence Hype: Managing Business Leadership Expectations,” Garner Research, June 5, 2018. 20 Oxford Reference, http://www.oxfordreference.com/view/10.1093/oi/authority.20110803095426960 (August 20, 2018). 21 Ransbotham et. al., op. cit. 22 Image courtesy of Yehia Khoja, Stanford Graduate School of Business Class of 2018, from forthcoming research paper, “Developing Corporate AI Competencies: A Strategic Framework for Successful AI Implementation.” This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. Noodle Analytics SM-301 p. 10 to generate business value with AI. The legitimate excitement about AI’s transformative power leads to unrealistic expectations.… The extravagant hype fuels a deepening skepticism.”23 Ajay Agrawal, Joshua S. Gans, and Avi Goldfarb, professors at the University of Toronto, framed the business opportunity of AI like this: Just as the semiconductor revolution and the rise of computer technology reduced the cost of arithmetic, AI presented an opportunity to take something that used to be comparatively expensive—prediction—and make it abundant and cheap.24 Data mining and pattern recognition across huge amounts of data could provide new insights— detecting anomalies, providing personalization, predicting events, and making probabilistic recommendations. AI, for example, could augment humans’ ability for dynamic pricing, predictive maintenance, and fraud detection. Because algorithms could “learn” and adapt based on new data, they could become more accurate over time. The global business value derived from artificial intelligence was projected to reach $1.2 trillion in 2018, an increase of 70 percent from 2017, according to Gartner, a research and advisory firm. And AI-derived business value was forecast to reach $3.9 trillion in 2022.25 John-David Lovelock, research vice president at Gartner, said: In the early years of AI, customer experience (CX) is the primary source of derived business value, as organizations see value in using AI techniques to improve every customer interaction, with the goal of increasing customer growth and retention. CX is followed closely by cost reduction, as organizations look for ways to use AI to increase process efficiency to improve decision making and automate more tasks. However, in 2021, new revenue will become the dominant source, as companies uncover business value in using AI to increase sales of existing products and services, as well as to discover opportunities for new products and services. Thus, in the long run, the business value of AI will be about new revenue possibilities.26 Managers had to find a strategy for bringing AI into their business. Lovelock predicted that between 2017 and 2022, enterprises would focus their AI efforts on niche solutions that addressed one need very well. “Executives will drive investment in these products, sourced from thousands of narrowly focused, specialist suppliers with specific AI-enhanced applications,” he said.27 23 See Gartner press release, “Gartner Says Global Artificial Intelligence Business Value to Reach $1.2 Trillion in 2018,” April 25, 2018, https://www.gartner.com/newsroom/id/3872933 (August 8, 2018). 24 Ajay Agrawal, Joshua S. Gans and Avi Goldfarb, “What to Expect from Artificial Intelligence,” MIT Sloan Management Review, Spring 2017, pp. 24-25. 25 “Gartner Says Global Artificial Intelligence Business Value to Reach $1.2 Trillion in 2018,” BusinessWire, April 25, 2018, http://markets.on.nytimes.com/research/stocks/news/press_release.asp?docTag=201804250930BIZWIRE_USPRX_ ___BW5453&feedID=600&press_symbol=169538 (June 14, 2018). 26 Ibid. 27 Ibid. This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. Noodle Analytics SM-301 p. 11 Competitive Landscape So who were these “specialist suppliers”? In 2016, the landscape of AI service providers was fragmented, with enterprises facing a variety of options including large systems integrators like IBM, management consultancies such as Deloitte, mid-sized service providers like Genpact and boutique operations like Noodle.ai. (See Exhibit 3 for a list of providers.) With AI being seen as a key competitive differentiator, some enterprises were reluctant to outsource their AI needs and instead were seeking to build capabilities in-house, sometimes starting with off-the-shelf tools and other times commissioning custom systems. The scarcity of AI experts was one constraint on the ability of enterprises to build their own AI capabilities from the ground up. According to a 2018 report in the New York Times, AI specialists with little or no industry experience could make between $300,000 and $500,000 a year in salary and stock. Element AI, an independent lab in Canada, estimated that in 2018, there were 22,000 people worldwide who had the skills needed to do serious AI research.28 Many headed to very large companies with deep pockets, like IBM, or to start-ups. AI-focused start-ups had raised $5 billion globally by 2016.29 The trend accelerated in 2017; according to CB Insights, equity funding for AI-related start-ups jumped to $15.2 billion in 2017, a 141 percent increase over 2016. And more than 300 AI-focused start-ups entered accelerators in 2017, a threefold increase over 2016.30 Revenue from data science and machine-learning platforms grew by 9.3 percent in 2016, to $2.4 billion. This growth rate was more than double that of the overall analytics and business intelligence market, which saw growth of 4.5 percent from 2015 to 2016. Still, data science and machine learning platforms represented just 14.1 percent of the total worldwide analytics and business intelligence revenue in 2016.31 Noodle.ai believed it would compete with a heterogeneous field of players. Some of these, like H20.ai, Domino Data Lab, Ayasdi, Microsoft Azure, and Google’s open-source TensorFlow, offered AI tools and platforms that companies could use to develop applications themselves. Others, like the India-based Mu Sigma, were more consulting driven. Firms like the Chicagobased predictive analytics start-up Uptake came closer to the SaaS model that Noodle.ai was pursuing, but Noodle.ai also saw similarities with companies like Palantir, which offered both software and services to clients. (In Palantir’s case, a large focus was U.S. government agencies.) 28 Cade Metz, “A.I. Researchers Are Making More than $1 Million, Even at a Nonprofit,” The New York Times, April 19, 2018, https://www.nytimes.com/2018/04/19/technology/artificial-intelligence-salaries-openai.html (August 20, 2018). 29 Brian O’Keefe and Nicholas Rapp, “Here are 50 Companies Leading the AI Revolution,” Fortune, February 23, 2017, http://fortune.com/2017/02/23/artificial-intelligence-companies/ (June 13, 2018). 30 Deepashari Varadharajan, “State of AI 2018 Briefing,” CB Insights, https://www.cbinsights.com/reports/CBInsights_State-AI-2018-Briefing.pdf (June 20, 2018). 31 Carlie J. Idoine, Peter Krensky, Erick Brethenoux, Jim Hare, Svetlana Sicular, and Shubhangi Vashisth, “Magic Quadrant for Data Science and Machine-Learning Platforms,” Gartner, February 22, 2018. This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. Noodle Analytics SM-301 p. 12 Noodle.ai anticipated that in the future, it might face competition from large enterprise resource planning (ERP) and customer relationship management (CRM) software firms, including SAP, Oracle, and Salesforce. Salesforce, for example, had launched its Einstein-branded AI capabilities, which include sales and service forecasting, in 2016; by 2018 the company said it was delivering more than 1 billion AI-driven predictions to customers daily.32 But Noodle.ai believed its relatively small size could present an advantage. “We believe we are faster and nimbler as a start-up to do things that a company like SAP can’t,” said Joshi. STRATEGIC LEADERSHIP CHALLENGES So what exactly, would Noodle.ai do? Matt Denesuk, Noodle’s chief data science officer, drew a distinction between what he called “mature AI” and “emerging AI.” Mature AI involved making systems that mimicked human intelligence, such as self-driving cars, or facialrecognition software, or computers that could understand natural language. In the realm of mature AI, the technical problems were well-defined and the business value was obvious. “No one is saying, ‘is there really a business case for a self-driving car?’ It’s self-evident,” Denesuk noted.33 In the realm of mature AI, progress was dependent on getting more and more data and taking advantage of the natural progression of computing power, and making small tweaks to algorithms. “Mature AI is more purely technical,” Denesuk explained. “You’ve got all the data, you know what the approach looks like, you kind of know the best algorithms, let’s just tweak the algorithm, let’s get more of the same data and just kind of keep turning the crank.” Denesuk knew Noodle.ai would not focus on mature AI, but emerging AI. Most enterprise problems, he felt, fell into this area, in which the technical problems were not very clearly linked to business problems. Data was messy; questions were ill-defined. He elaborated: You walk in, and you don’t know what the problem is yet that you want to solve. The data that you have is often very poor quality, or the data capture policies are set up for something else—business intelligence, SEC compliance, financial reporting. There may be a steel mill that has sensor data, but they throw it away every month. Each company will have a different data format, different data capture policy, different data models. What tends to drive success is what you focus on: really understanding the business problem, and what’s the right technical problem to solve. That requires knowing where do they have good enough data to solve that. It’s that intersection of where the data is, and where the business problem is. Investments are very tentative and small. These businesses want to see return in three months or something, and they only want to invest a little bit upfront. So you really have to kind of do a dance where you can show a little bit of value with a little bit of investment and kind of go forward. 32 Blair Hanley Frank, “Salesforce Einstein Now Powers Over 1 Billion AI Predictions Per Day,” Venture Bea,. February 28, 2018. https://venturebeat.com/2018/02/28/salesforce-einstein-now-powers-over-1-billion-aipredictions-per-day/ (June 13, 2018). 33 Interview with Matt Denesuk, May 11, 2018. All quotations are from this interview unless otherwise noted. This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. Noodle Analytics SM-301 p. 13 Pratt, Joshi, and the rest of their early team knew they couldn’t be a pure consulting business; they would have to figure out how to make a product that they could sell to many customers, perhaps with customization on top. In its approach, Noodle.ai was not alone. A survey by Gartner of 70 AI projects conducted by 24 AI-focused service providers found that 80 percent of companies had developed some sort of platform to anchor their offering.34 Some providers licensed the platform to clients, while others provided it in conjunction with their services (without separate fees for licensing). Some were proprietary, while others were well-known third-party options. For client companies, prebuilt solutions promised a rapid timeframe to seeing bottom-line impacts, while subscription models could lower the upfront costs. Most engagements included trial periods. Given the nascent nature of the market, companies in this space took on a wide variety of clients as they explored which industry niches they were most suited to serve. Gartner noted that “service providers have been primarily opportunistic in terms of their… engagements, and their track records are spotty in any specific process, domain and industry.… Buyers cannot readily rely on track records in a functional or industry-specific scenario.”35 Noodle.ai began a period of rapid exploration as they built their team, worked on a product strategy, and homed in on industries where they thought they might get the most traction. One hurdle Noodle.ai would have to overcome as it approached clients, said Denesuk, was that large enterprises were nervous about “outsourcing” AI rather than developing it as a core competency. He explained: People see this as so strategic and they say, “Well, you know I should be doing this internally.” They’re like, “Maybe you can kind of just help us get started. We want you to train our people.” And we don’t mind. We’ve said, “We’ll work with you and help you and get your skill up as long as there’s continued business for us and there’s so much opportunity there should be.” But it’s not an effective business model for us to [just] say, “Look we’re going to invest in training your people,” because that’s not going to scale for us. North Stars in Developing the Business Model Pratt and Joshi had several guiding lights as they worked to get Noodle.ai off the ground. One was their extensive experience consulting for major enterprises. “We know the Fortune 500 globally and in the U.S.,” said Joshi. “That is our core competency.” Second was to focus on being a product company. Joshi saw that product companies had higher valuations than services companies. “A service company will get two times revenue. A product company can get six to 10 times annual recurring revenue and in the world of AI perhaps even 34 Susan Tan, Neil Barton and Frances Karamouzis, “Market Guide for AI-Related Consulting and SI Services for Intelligent Automation,” Gartner, May 23, 2018. 35 Ibid. This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. Noodle Analytics SM-301 p. 14 higher,” Joshi said. “In a product company, it’s not the revenue that you are making today but the revenue you will make in the future; the product is the intellectual capital. In a services company, you are the intellectual capital.” A third guiding light was the focus on helping customers realize business value. And they had first-hand experience in this (via the Value Realization Method Framework) from their Infosys Consulting days. Joshi believed many consultants lost sight of the value they were actually delivering for customers. But product companies could fall into the trap, too. “Most people focus more on what’s my technology—how cool is it? How complex is it? They don’t focus on what value they are providing to customers.” He said, “To me, it is actually fairly simple. Whatever your product or service is, how is it going to help your customers either increase revenue or reduce cost or both?” The Value Realization Method was a disciplined way to quantitatively link programs to the creation of shareholder value on one side and to customer value on the other. 36 Take, for example, a company that took 12 days to process an order for shoes. Using the VRM framework, if the company was now able to ship the shoes in six days, then effectively the company had achieved a 50 percent improvement in its business process. By focusing on process metrics, VRM made processes more efficient.37 Explained Pratt in 2011: A lot of consulting firms would just go forth and use their methodology to implement a technology solution. If a client asks us to implement a new technology, we will ask the client about their business case and say, “Well, for us to do the project right and make sure that we optimize and increase your value, we need to understand the business logic behind why you are doing the program and, specifically, what process metrics are you trying to change, and to what extent.”38 Pratt and Joshi knew from their IC experience that many clients did not have a detailed understanding of their own operations; companies might have high-level operating metrics but not the key metrics at the detailed level to understand if the company was operating efficiently. Coming up with the correct metrics would help Noodle.ai focus on what problems it could solve for clients. Pratt drew a simple Venn diagram to illustrate his point. One circle (blue) was everything that was possible in data science. The other (red) represented things that added value to a business. AI experts were focused on the blue circle, Pratt said, while consultants were focused on the red circle. Noodle.ai, he explained, was trying to home in on the purple overlapping section. 36 37 38 For more information see “Infosys Consulting in 2011,” GSB No. SM-195, June 1, 2011, page 2. Ibid. Ibid. This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. p. 15 Noodle Analytics SM-301 Source: Steve Pratt. “Our business model is: 1, we understand which hypotheses are very likely to work in your business; 2, which are creating business value; and 3, which are something you can actually use in your business,” Pratt said. Core Competencies At Infosys Consulting, formulating these client hypotheses fell to consultants like Pratt and Joshi. At Noodle.ai, the two knew that they would need a team of people with expertise in both data science and business. Pratt and Joshi thought about the core competencies they would need at Noodle.ai. As they worked with some initial clients, they discovered it would take six different skills to build an effective AI solution: 1. Data Scientists — to understand data and develop algorithms 2. Data Engineers — to “cleanse” data and get it into usable formats 3. Software Engineers — to convert the data science code into industrial-strength software code and connect these applications with clients’ existing architecture 4. Infrastructure Engineers — to deal with the supercomputers powering the code 5. User Interface Designers — to design dashboards and other easy-to-understand systems that make it simple for employees to use (adoption, said Joshi, is half the battle) 6. Business Consultants — to understand clients’ business problems and develop the framework technology solutions Noodle.ai itself would also need a sales team. Pratt quickly realized this combination of skillsets would make Noodle.ai feel much different than Infosys Consulting. And that presented a new set of management challenges. He recalled: One of the exciting things about Noodle.ai that’s very different than Infosys Consulting is the culture. In a consulting business, it’s a bunch of people who skew toward being type A extroverts. At Noodle.ai, it is very heterogeneous. This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. Noodle Analytics SM-301 p. 16 You have data scientists that typically come from an academic background. They’ve been doing research, and are actually more motivated by publications and breakthroughs. You have the computer scientists, who are typically more introverted, and the computing [infrastructure] engineers or high-performance computing experts, who are very introverted; they will retreat into their caves if you don’t drag them out. Then you have user interface designers, who are very artistic, and all the wonderful things that go with that. And then you have the consultants who are the type As. Finally you have salespeople, who we didn’t really have at Infosys Consulting. Salespeople are super extroverted. And so you can imagine getting all these people to work together is very important. Adding to the complexities, Noodle.ai would operate out of multiple offices: its headquarters in San Francisco; a secondary office in Palo Alto, California; and a third location in Bangalore, India, that opened in August 2016. Headcount was expected to be divided about equally between the United States and India. FIGURING OUT THE PRODUCT Pratt and Joshi chose Noodle’s name in part because it didn’t lock them into a particular industry; as they grew, Noodle’s initial clients and projects were, to say the least, a motley bunch. In addition to XOJET, Noodle.ai worked with a customer on a project to use social media data to help identify potential extremists. They worked for a tire distributor, an airline engine maker, an apparel company, and a steel plant. The idea was to explore various sectors, then figure out which ones offered the most opportunities. Big River Steel was a company in the TPG portfolio. Its CEO, David Stickler, met Pratt at a TPG conference, and they started to discuss Stickler’s vision for a “smart steel mill” that would absorb all the data at its disposal, analyze it, and use those learnings to improve operations. As the Noodle.ai team quickly found, there were myriad data sources that could be wrangled to address different business, safety and environmental questions for the $1.3 billion, 1,300-acre mill in Arkansas. They gathered historical data from the mill, external industry and market data, and real-time data from sensors in the plant itself.39 For one project, Noodle examined the financial spread between scrap and finished steel; Big River Steel wanted to know if it could find a mechanism to hedge on scrap steel. Another project involved harnessing data from the 50,000 sensors in the facility to improve operations and reduce dangerous accidents known as caster breakouts. A third effort looked at how to minimize energy usage by looking at production schedules and taking advantage of different electricity rates at different times of day. Denesuk recalled: 39 Rachelle Blair-Frasier, “Q&A: Inside a Smart Steel Mill,” Manufacturing.net, January 4, 2018, https://www.manufacturing.net/news/2018/01/q-inside-smart-steel-mill (June 18, 2018). This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. Noodle Analytics SM-301 p. 17 One application we built for them showed them predictions, one day ahead, two days ahead, three days ahead, hour-by-hour, how much electricity they were going to need for their production schedule. You input the production schedule, and they can identify the spots when they can sell electricity back to the utility. The more advance notice they give the utility, the better price they get. This is just cash in their pocket that goes right to their bottom line. Building The BEAST Noodle’s early work with XOJET, Big River Steel, and a few other companies helped the team get a sense of what their core AI product offerings might be that would apply to many companies, such as demand forecasting and predictive maintenance. They realized that they might have opportunity not only to serve end producers like Big River Steel, but also to push AI to companies further upstream—for example, the companies that designed and made steel mills. Joshi remembered: Demand forecasting was our number one application. But we said, “We can’t just be a demand forecasting company.” Forecasting is not a new concept. A lot of companies use statistical techniques but not machine learning techniques for forecasting. The difference with AI, of course, is once you build the model, as new data comes in, it changes because it’s not explicitly programmed. That’s the machine learning part. And so it is a living, breathing beast, so to speak. Ted [Gaubert] was describing this to me, and I said, “What are you going to call this beast?” And he looked at me and said, “The Beast,” and it stuck. The BEAST, as Noodle’s core product was christened, would be the company’s development platform, encompassing hardware (a supercomputing nucleus), a software platform, data science tools and algorithms, as well as curated data sets or cartridges unique to various industries and customers. Ninety percent of the platform would be modular, reusable components common to all customers, while 10 percent would vary by customer and industry segment. Pratt reverseengineered “The Beast” name into a recursive acronym, with the letters standing for “BEAST,” “enterprise,” “artificial intelligence,” “supercomputing,” and “technology.” The BEAST was built not on the cloud but on an in-house supercomputer put together by Gaubert and his team. (Noodle’s management had decided that there was no cloud offering that they could take advantage of that offered sufficient cost efficiency.) Noodle.ai kept the computeintensive portions of its AI work, including the training of algorithms, on its private system. The less intensive portions were deployed to AWS, Microsoft Azure, or Google Cloud. Noodle.ai believed that at the rate it was growing, it could recoup its investments in hardware in two to four months. This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. Noodle Analytics SM-301 p. 18 The BEAST was up and running by December 2016. On March 14, 2017, on the company’s first anniversary, Noodle.ai publicly announced the general availability of The BEAST, giving it the tagline “The Brain Inside Your Business.”40 Pratt conceived of The BEAST as “a library of AI applications applicable to the enterprise.” It had 10 modules: five focused on supply-side issues (materials, production, logistics, warehouse, and inventory) and five on demand-side issues (pricing, product, promotions, sales, and customer). Each module would come with interconnected applications; for example, to help with better demand prediction, optimizing inventory allocation amid volatile demand, anticipating in-bound materials risk, predicting production delays, or optimizing labor planning.41 Each module had three “engines”: a sensing engine (what’s going on) a prediction engine (if you do X, Y will happen), and a recommendation engine (here’s what you should consider doing given this situation). To go from raw data coming in one side, to a finished application on the other side, Noodle.ai had to work through four discrete steps. First, it had to gather and prepare the data to teach the algorithm, which could be a laborious process. Pratt explained: For instance, if you have weather data and demographic data, and steel price data, and sensor data, they all have different frames of reference of space and time. Some might be zip code, some might be latitude/longitude, some might be a street address, some might be whatever. From a time perspective, some might be by month, some might be by quarter, some might be by millisecond. And so getting all those data to align is very complex. And usually there’s some nonsense data, so you have to clean the data, integrate the data. That’s what we call the teaching layer, is getting ready to teach the algorithm. In this phase, Denesuk explained, data scientists would be working interactively to develop their models. They would analyze the data, visualize it in different ways, and look for the strength of relationships between different variables. Second was the learning step—feeding data into the models so they could “learn.” The mathematical models could be exceptionally complex. “We’re not dealing with two-dimensional linear regressions,” said Pratt. “We’re mostly dealing with N-dimensional, 100-dimensional nonlinear regressions.” Data scientists would look to see how well the models predicted outcomes based on the data they were given. 40 “Noodle.ai Unveils BEAST Enterprise Artificial Intelligence Supercomputing Technology – Powered by NVIDIA DGX-1 AI Supercomputer,” BusinessWire, March 14, 2017, https://www.businesswire.com/news/home/20170314005324/en/ (June 18, 2018). 41 Srishti Deoras, “Deepinder Dhingra Of Noodle AI Wants to Revolutionise Organisations With A 10X Approach To EnterpriseAI,” Analytics India, June 12, 2018, https://analyticsindiamag.com/deepinder-dhingra-of-noodle-aiwants-to-revolutionise-organisations-with-a-10x-approach-to-enterpriseai/ (June 19, 2018). This document is authorized for use only by Tristan Post ([email protected]). Copying or posting is an infringement of copyright. Please contact [email protected] or 800-988-0886 for additional copies. Noodle Analytics SM-301 p. 19 Third was a “thinking” step, where Noodle.ai data scientists would seek to improve the model based on the results of step 2. Finally, there was a communicating step, which involved delivering the model’s results to the customer in an easy-to-digest fashion or interface. Nehal Raj of TPG summed up the business model: We have proprietary technology. Back-end is this thing called The BEAST. It’s very much a technology and software business model that’s leveraging real IP written by a development team. But we also have data scientists on staff who, when we sell a new project or engagement with a customer, can sit alongside the software and help the customer achieve their goals. And so, we have very much a blended softw