Managing AI Projects PDF

4 Managing AI Projects Understanding the AI Project Whether you are managing a traditional project or an AI project, you need to learn AI. As per Andrew Ng, an AI influencer, AI is the new electricity, and without electricity, you cannot function and progress rapidly. It will tra...

4 Managing AI Projects Understanding the AI Project Whether you are managing a traditional project or an AI project, you need to learn AI. As per Andrew Ng, an AI influencer, AI is the new electricity, and without electricity, you cannot function and progress rapidly. It will transform every industry and create massive economic value. It is excellent at automating tasks and will have an impact on every sector – from healthcare to manufacturing, logistics, retail, and many. Humans not only have to manage both people and projects, but they also struggle to com- plete the projects within budget, scope, and time. The emergence of the project manage- ment field and industry is a collection of people and processes to achieve this ultimate goal. Finding a way to streamline the process and increase efficiency is the major objective of project management. Project managers commonly manage multiple teams as well as various projects at the same time. These projects usually must work together to achieve bigger goals. Project managers can produce more reliable results and help individuals guide them more effectively by creating a smoother process. Artificial intelligence is increasingly used in project management tools and technologies to handle everything from scheduling to analyzing a team’s work and providing recommendations. These augmented tools make AI a clear advantage for future project managers. The sum of AI and people should always be greater than the sum of entire AI components. As ® 167 Chapter 4  Managing AI Projects machine learning (ML) and AI development, new projects have rapidly emerged with different problems and requirements. With the rise of AI technologies, it is no longer a “nice to have” technology. In fact, technical project managers need to maintain a healthy relationship with these concepts. World Economic Forum report states that 97 million new jobs will be created and 85 million jobs will be replaced by machines with AI by 2025 due to AI. As artificial intelligence has disrupted industries from finance to healthcare, the technical project manager who grasps this opportunity must understand the uniqueness of AI project management and how to best prepare for the changing situation. What is a Project? As the Project Management Institute (PMI) defines, the project is a temporary endeavor with a definite start and an end date and therefore defined the scope and resources. A project is unique because it is not a routine operation but a specific set of operations designed to achieve a single goal. In other words, a project is nothing but a series of tasks that requires completion within a specific period. To achieve a specific result, projects can also be defined as a set of inputs and outputs needed to achieve the goal and could be both complex and straightforward that could be managed either by one or 100 people. What is an AI Project? An AI project is similar to project definition (PMBOK [PMI 2017]) except that AI projects are more dependent on data and algorithms. Availability of initial data for training, continuing data collection strategy, cleaning up the collected data, determining the valuable features of data, transforming data to fit a model, selecting appropriate algorithms, evaluating multiple algorithms to determine accuracy, comparing against other algorithms, and determining the learning rate of the model is the core activities of AI project management. These models may function autonomously; however, they need human intelligence to speed up the learning process. The first phase of the AI project is most important. It defines and identifies business cases and use cases like regular projects, but its risks are more significant. The companies learn from each other’s mistakes, challenges, and new knowledge to monetize it because the AI project is still in the discovery phase and the process is constantly evolving. Al project value propositions include excellent business value, causing an abrupt increase in revenue in the shortest time possible by gaining more customers to increase market share in the mode of start-up companies. AI projects are continuous, collecting a new set of data and applying predefined/preselected algorithms and pre-trained models that have already gone through the initial training. The goal is to reach success with high accuracy, about 80 percent or more. The required accuracy is based on the business case, the project’s goal, and the problem. For example, the Al self-driving car project needs about 100 percent accuracy and has zero fault tolerance. This is because human safety is directly involved in a few of the many cases. But some other AI projects, such as assistance provided by Amazon’s Alexa project, may not need 100 percent accuracy. In general, more accuracy with less fault tolerance is better. ® 168 Chapter 4  Managing AI Projects Al projects need the latest trends, demanding roles, and skillsets such as data scientists, data architects, data designers, data engineers, ML engineers, Al engineers, Al consultants, Al scientists, cloud engineers, and subject matter experts in their respective fields. In addition to human resources, machines are also part of the resources needed, such as IoT devices, virtual reality devices, and robots. The “artificial intelligence” term is used to define machines or computers that mimic human’s natural intelligence to solve complex problems and foster learning. Some of the advanced artificial intelligence programs’ tasks include Natural Language Processing, Deep Learning, and Computer Vision. Before continuing further, the first obvious question is, “Will Artificial Intelligence replace project managers?” And the answer is no, no chance! Artificial Intelligence (AI) will not replace humans, and in fact, it is a work augmentation tool. Artificial intelligence cannot manage a project by itself, even the smallest project. As mentioned above, artificial intelligence on its own is not that good at performing subtle tasks. For instance, AI can significantly improve your tedious status reports and messy resource scheduling, but it cannot collect requirements or get stakeholder buy-in. As an influential and effective project manager, your job is safe. As with every technology, AI alone cannot guarantee success. Numerous project management tools are available, and various companies plan to upgrade them to a better and easier way to manage projects. This will confirm the strategic value of project management. Is AI Project Better Than Traditional IT Project? It depends on a specific situation. Some projects do a better job of including AI, while other applications become unnecessarily complicated due to AI technology integration. Eventually, it all depends on the use case and its value in artificial intelligence (AI). Benefits of Artificial Intelligence (AI) in Project Management AI’s power lies in its ability to apply human intelligence without putting a biological and emotional burden on real people. AI does not require rest, does not distract, and can interpret millions of information points simultaneously. AI technology can aggregate task status in project management software to generate weekly status reports, calculate the budget impact on increasing scope and timelines, and perform risk modeling. Here are some of the various benefits of AI-enhanced PM tools: 1. Automate tasks: Automate repetitive, tedious tasks so you can spend time on vital problem-solving and beneficial tasks. Gartner predicted that, artificial intelligence enhancement would generate US $3.9 trillion in business value and restore 6.2 billion hours of employee productivity. One of the reasons AI is gaining popularity amid patrons is that it cares about monotonous, repetitive tasks. 2. Historical data: Historical data can be used to perform calculations and predictions to improve the results’ accuracy. If programmed, AI will always refer to previous project results for prediction and estimation. A project manager may not access the results of other projects/things (historical data) for reference. 3. Risk modeling: Risk modeling and analysis are based on scope, available resources, and budget reduction, among many other things. This is especially useful when agile project management technique continues to lead the way ® 169 Chapter 4  Managing AI Projects projects are run. Unpredictable changes will always occur, but artificial intelligence will forecast their likely effects based on how similar changes have affected earlier projects. 4. Decision-making: Artificial intelligence helps speed decision-making through a process-based rule. AI programs only follow specific, rule-based workflows. This means that when AI monitors and sends notifications about task status and updates, obstacles and bottlenecks can be resolved quickly. Resource scheduling and allocation: Figuring out issues like people required to perform a specific task, and if they are available, whether they will be able to complete the tasks are all difficult issues. Figuring out whether the people needed to perform a specific task, if available, and complete the task are all difficult issues. However, it can suggest your project’s best resource allocation if you can load the necessary information into the AI-enhanced project management tool. This is a powerful advantage of AI, which can help the project manager optimize resource scheduling and allocation. AI Tools Can: Evaluate the type of resources required by the project based on the required tasks, such as building a custom workflow and then accomplishing QA testing and many more. Use historical data to estimate the duration of the task. Use the database of people’s skills to help you choose the most qualified candidate for required tasks. Examine the workload and availability of individuals who can contribute to the project. Determine how many tasks a person can do based on weekly productivity metrics. Compare the proposed resource plan with historical data to find inconsistencies and improve the proposal’s accuracy. Propose the best resource plan with the available team. AI can save you from the troubles, obstacles, and uncertainties of completing all these steps independently. But this is only possible if the system is kept up to date and has the correct information. Difference between AI Projects and Traditional IT Projects Artificial intelligence projects require a different approach from traditional IT projects to succeed. The key distinctions between AI and traditional IT projects are listed here to help you choose the right strategy. 1. Solution Approach The traditional IT development process is for a designated solution. Whenever it is difficult to determine a solution, the result becomes uncertain and full of risks. This type of development is top-down programming. For the proof of value (POV) of AI projects, a bottom-up approach will be adopted. Machine learning technology starts from the bottom to process data until the solution. ® 170 Chapter 4  Managing AI Projects Let the computer conclude its own rules based on a series of attempts at the data set. In this case, artificial intelligence concludes its own rules and processes of working with a large data set. Compared with applications, artificial intelligence should be regarded as a resource that develops over time. 2. Visible Beyond The development of an AI is exploratory, mainly focusing on getting to know the data you have to work with because it is the data at the heart of an AI application. It is a matter of testing different appropriate methods for this type of task, adapting them to the data you have, and evaluating their performance. As a result, patterns are often found in the previously hidden data, and thoughts are awakened about new ways of managing the data. 3. Predictive Project Development The project is predictively conducted, and project design supports predictive project development. AI is a new development landscape that opens many new opportunities that are often not known in advance. It is about analyzing ongoing developments and events when new knowledge and insight results occur. The followings are few main points for project development: - The project has to support working on both short- and long-term tasks simultaneously, agile short deliveries and test shots with defined but flexible project goals, and a long-term vision that evolves as the project progresses. An approach that delivers the best short-term results for the money and supports long-term effects allows test shots to generate lessons learned during the project. A traditional project methodology based on predefined goals and broken-down tasks increases the risk of not getting possible effects. AI projects can lead to, for example, results of 100% and much more than that. 4. Change Management Cognitive AI solutions can have a significant business impact directly in order to have effects; from the beginning of each development step, it needs to be planned for how resources are redeployed. The process of spending a few days or weeks managing a person may become an event. What these resources should start to do needs to be planned early. It will be important to start thinking ahead and see what opportunities or work you have for them? Predictive project development is different from traditional application projects. Predictive projects develop over time; you cannot “ manage change “ this type of project because the result is not always known in advance. A project methodology that governs a high level and common sense is required in order to be able to control and follow up on this type of project. As the cycle matures, the development prospects of artificial intelligence will often bring some opportunities. Although this method’s results are almost always high-yield friendly, it usually has higher development costs and longer development time. This means that it must cross several exploration stages and hits and trials if a project is to be completed. ® 171 Chapter 4  Managing AI Projects Why do AI Projects Fail? Multiple factors come into play when implementing an AI project. Data, skill, domain understanding, culture, AI strategy, Data strategy capabilities are a few that play a vital role in AI projects’ success. Enlisted below are some of the numerous reasons why AI projects fail to deliver: 1. Expectation Misalignment Usually, due to the imbalance expectation, most AI projects do not see the light of the day. The root cause of AI’s business challenges is often the result of increased short- term expectations of technologies that inherently operate in long-term mode. In enterprises, if AI-based solutions are not accurate enough to meet different user perception needs, you can see the following situation where expectations are inconsistent, for example, in the case of music streaming applications, suppose your AI suggested that “Next Song” is not exactly what the user thinks belongs to this genre. This is why companies often use the word “possible” when displaying products or services that their users may be interested in. 2. Inefficient Data Management AI tends to make the wrong decisions based on faulty data sets. When the data is incorrect or incomplete, problems arise in the AI project management solution, meaning; it is not ready to fit the AI model. For the AI system to work as anticipated, there must be refined data for the system to learn and analyze patterns. When building data sets that support AI, the focus should be mainly on dividing structured and unstructured information according to modern data collection strategies. 3. Poor Data Governance Garbage in and garbage out still apply. Most companies underestimate the importance of quality data for the success of AI implementation. A few companies have poor data governance and poor data hygiene habits, leading to data being suspected, duplicated, or used elsewhere. Skewed data samples may also pose risks. For example, this can cause overfitting problems, resulting in incorrect output when running in production. Artificial intelligence systems will only learn from their data. Therefore, there is always a risk of human bias being propagated through machine learning. Establishing master data management and governance and develop a central data repository (a data engagement platform or data lake). “To create transformative AI solutions, we need a holistic, synergistic, and simultaneously integrated flow of information,” Seth Earley, author of “The AI-Powered Enterprise: Harness the Power of Ontologies to Make Your Business Smarter, Faster, and More Profitable.” Earley says that a consistent representation of data and data relationships that can inform and power AI is “the master knowledge scaffolding” for AI-driven transformation. 4. AI Scope Creep The most common reasons for scope creep in machine learning projects are trying to do many things simultaneously and underestimating the work required to prepare the data. To overcome this obstacle or solve this problem, stakeholders first need to be ® 172 Chapter 4  Managing AI Projects managed to understand the benefit of starting with a small win compared to a grand plan. As you build and test, continue this communication throughout the project. Start with small-small features that can easily be well-defined and tested. If you have a complex task, try to break it down into simple tasks, which can be a good substitute for your main task. 5. Machine Learning Expectations Every project manager (PM) should consider the user experience of the AI products they create and how best to manage the team that builds them. Google released a fantastic post explaining their philosophy regarding UX and AI, emphasizing human engagement. This is especially important if your machine learning product has to interact with the user or even be replaced by the user. The design should put users and system operators under the least amount of stress possible. For instance, chatbots can easily be replaced by a human operator while often being based on machine learning. Stakeholders may also expect machine learning products to exceed what they can provide. When writing about AI products, the media frequently creates a lot of hype. The PM must therefore establish reasonable expectations. Make sure to explain what an AI tool is and can be implemented for your stakeholders so that you can fully manage their expectations before they test the tool. A good user experience is excellent, but it cannot provide value for users who expect unrealistic results. Therefore, managing and promoting AI and educating its stakeholders about AI and its realistic capabilities are important for any PM involved. 6. Reality Check Despite the popularity and broad prospects of AI, it is likely that the problem you want to solve does not require complex AI solutions. It is simple to use, and, in most cases, more reliable statistical regression models can solve many prediction problems. The project manager needs to perform an integrity check before starting the project to ensure that it requires AI or ML. It is sometimes practical to start with a simple statistical model and develop parallel with machine learning-based solutions. For instance, if you want to develop a recommendation engine, it is wise to start with a simple solution with a faster development life cycle and provide a good benchmark for subsequent ML models. 7. Quality Assurance (QA) And Testing In ML Artificial intelligence is a fairly new field, and never before have so many applications used deep learning to achieve their goals. These new developments have brought a series of challenges, especially in testing. Although it is relatively easy to test standard software with an explicit “ruleset,” it is challenging to test in detail machine learning models (especially models built using neural networks). Currently, most data scientists test ML models themselves; however, few test methods agree with the standard QA team (as per the project requirement) to ensure that ML products don’t fail in unexpected ways. ® 173 Chapter 4  Managing AI Projects 8. Talent Shortages The current demand for world-class AI experts is fierce in obtaining suitable talent. Hiring and retaining these much-needed technical experts has become the focus of many organizations. When looking for AI experts to join the team, a project manager pays close attention to AI dynamics because they may affect your recruitment cycle, budget, or the quality of the work done. This shortage is beyond the scope of innovative thinking to create new deep learning algorithms, as is the case for high-quality data engineers and scientists. 9. Legal and Ethical Challenges AI in project management has legal and ethical challenges. The initial set of challenges comes from the data used to train ML models. You must know the source of the data you use, especially whether you have the right to use it and the data’s license. Consult your lawyer to resolve such issues before deploying a model that trains on data that you may not have the correct license type. As this is a relatively new field, many answers are unclear, but project managers should ensure that their teams use only the data sets they are authorized to use. Finally, check whether your data can be easily used with existing ML methods rather than inventing new ML techniques. The project’s duration and scope will expand dramatically if everything is done from scratch. Please note that if you try to solve an ML problem that has not yet been solved, it will likely fail. Despite machine learning and many research papers, solving machine learning problems is still a challenging task. It is always easy to start in the ML field with many good examples of algorithms and improve it rather than invent new things. Pillars of AI Project Success Organizations worldwide are now shifting their business strategy towards emerging technologies like artificial intelligence and machine learning. The secret to an AI project’s success lies within pillars like technology, business, people, and data. AI is able to improve customer experience in its entirety only in the presence of the following pillars. 1. Technology When people talk about AI (usually), they talk about AI hardware or AI software. The most significant recent leap in artificial intelligence has come from advances in hardware. The chip has extraordinary processing power supported by AI algorithms. The Internet of Things (IoT) records a large amount of data that AI systems can process. High-bandwidth networks can provide high-speed round-trip data at a low cost. Advances in software have been brought to this hardware. While the theory and algorithms behind deep learning are decades old, the recent hardware advancements and high network bandwidth make it more practical and beneficial. Most of the AI algorithms are part of open-source projects. Thus, AI software is almost free. On the contrary, hardware requires physical machines, energy, and, thus, money. Therefore, the organization’s ability to acquire technology is financial resources (monetary investment). ® 174 Chapter 4  Managing AI Projects 2. Business Artificial intelligence is a problem-solving tool that uses past data to solve future problems. So, an organization should invest only keeping in mind if: the organization has enough data the AI solutions benefit the bottom line of the organization the mix of data and business awareness determines whether an organization has a business case for using AI. 3. People Technology and business opportunities will not be self-explored and utilized. Organizations need skilled human resources to perform this discovery and development process. Identifying issues that may benefit the bottom line of the organization requires a lot of domain knowledge. These problems need to be changed into a form that AI technology can solve. Relevant technical pipelines need to be placed to solve these problems. The entire system needs to be monitored, maintained, and continuously optimized. Organizations without one or more of these pillars are unlikely to succeed in AI. Let’s understand in detail the impact in the organization if one of the crucial pillars is missing. 3.1 Technology + Business - No People = The Outsourcer These companies have considerable financial resources that enable them to invest in technology. However, they also encountered new business problems and required AI data. Unfortunately, they do not have the expertise of the people to build and utilize AI solutions. Therefore, these companies outsource the people aspects of the AI system to a third party. This approach faces two serious challenges: Third-party never understand the business in detail and hence have incomplete domain knowledge and imperfect incentives. This is the reason why their solution is not the best. The third party may steal the organization’s data and sell it to competitors. “Blind off-the-shelf user” is a variation of “the outsourcer.” These organizations thrive because of the AI platforms from Google, Microsoft, and Amazon. Unfortunately, the dangers precisely remain the same, and there is no heed to the consequences. 3.2 Technology + People - No Business = The Enthusiast Enthusiastic organizations are like outsourcers because they have financial resources. They also understand the importance of hiring talented people, for instance, machine learning engineers, data scientists, and domain experts. Unfortunately (unlike outsourcers), these companies do not have a business case for AI. 3.3 Business + People - No Technology= Poor man These kinds of organizations have a compelling business case for AI and people. However, they do not have enough finances to invest in technology. ® 175 Chapter 4  Managing AI Projects Each of the above examples lacks a pillar of AI. Many companies lack more than one like below: Outsourcing enthusiasts (Only technologies): These people have invested in advanced technology without considering people or Businesses. Unexplored opportunities (Only businesses): These organizations have an AI business case but do not have people or resources to invest in technology. AI Club (People only): These organizations have neither a business case for artificial intelligence nor a business case for technology; however, they have a group of interested people, even experts in artificial intelligence. For these people, it is more like a social club than a more impact-making organization. A successful AI project requires all three pillars: technology, people, and business. If the organization lacks any of these pillars, the AI project will be at risk. It either may not see the day of the light or will not be completed on time. Issues in AI – real and not real There is really a lot of hype going on right now in the intersecting worlds of artificial intelligence and robotics. And a lot of is just hype and exaggeration. ® 176 Chapter 4  Managing AI Projects One common myth is that robots are taking jobs away from people. In truth, robots and automation free up workers to do more productive tasks. The truth of this can be seen in job statistics – unemployment in the US is at a 20-year low, despite massive improvements in factory automation. The fact is that the improved productivity of robotics creates more jobs than it removes. I’m not here to say that individuals are not losing manufacturing jobs, but I do say that the overall level of employment has increased, not gone down as a result of automation and increased productivity. I do recognize that the modern worker, even someone like myself, who works in technology, must be ready and willing – at any age – to retrain themselves and to learn and adapt to new ways of working, new economies, and new opportunities. I’ve had to completely retrain myself at least five times as new markets were invented ® 177 Chapter 4  Managing AI Projects and new technologies have emerged. Sometimes there is a “second wave” where some technology was invented but then disappeared when it was too expensive for the benefits it provided, or the proper hardware had not been invented yet. Neural networks fit into that category, as does virtual reality, which was a big deal in 1999, and has now re-emerged with very small high resolution screens that were developed for cell phones. I’m quite interested in the long-term impact of what has been called the “sharing economy”, where companies like Uber, Lyft, and AirBnB create value by connecting suppliers and consumers on a massive scale without owning any of the capital or resources to actually provide any services. All of this is enabled and made possible by the ubiquitous internet, which continues to grow and evolve at a rapid pace. I often use the term, “but that’s a decade in internet years”, referring to some idea that is maybe 24 months old, to indicate the rapid turnover in internet tech. This trend will continue. It will be interesting to see if anyone owns a car in 20 years, or only a subscription to a car service. Another trend that has become very interesting is the lowering of barriers to entry in a lot of businesses. You used to have to have an enormous machine shop and giant machines to make plastic parts – before 3D printers came and put that capability on your desktop. Want to make movies? You can do it on an iPhone. Want to start your own recording studio? The parts for professional results (with a large amount of effort) are available for less than $200. One item that definitely fits into that category are drones, or small unmanned aerial vehicles. When I started making unmanned aerial vehicles, or UAVs as we called them, a decent GPS and IMU (inertial measurement unit) – the things that make unstable quadcopters possible to control– cost tens of thousands to hundreds of thousands of dollars. The real breakthrough in drone technology did not come from aviation, but rather from your cell phone. The developments in cell phones enabled companies to make billions of dollars making the next cell phone or smartphone, or hand-held computer pacifier, or whatever you would want to call it. The convergence of very small radios, very small GPS, and very, very small accelerometers, enabled an entire world of unmanned flying things to emerge. That, along with higher density batteries that came from (you guessed it) cell phones and laptops, allowed people to discover that if you put enough power on it, you can make almost anything fly, including you. The secret to the flying quadcopter’s amazing success is that the tiny accelerometers (that measure changes in movement) and tiny gyroscopes (that measure changes in angles) became cheap and readily available. Without these sensors, and the robotics algorithms that control them, quadcopters are unstable and impossible to control. Another reason for the quadcopter’s success is that it uses only the throttle setting – the speed of the motors – to control all of its aspects of flight, including stability. This compares with the very complicated collective controls and cyclic pitch controls that make a helicopter work. You can see the difference in an R/C helicopter, that is very expensive, and only a few people can fly, and a quadcopter, that is quite cheap and can be flown by anyone, with the help of a computer and some sensors. ® 178 Chapter 4  Managing AI Projects Understanding risk in AI One subject I talk about frequently at conferences and in print is the risk of artificial intelligence in terms of trust and control. I’m not talking here about AI running amok, but rather with AI being dependable. It is quite interesting that the sort of AI we have been doing – specifically, artificial neural networks—does something very few other computer software do. Given the exact same inputs and conditions, the output of an AI system is not always the same. Given the same inputs, the AI will sometimes come up with a different answer. The formal name for this behavior is non-determinism. There is a second corollary to this. Given the same inputs, the AI process will sometimes take a different amount of time to complete its task. This is simply not normal behavior for a computer. We have gotten used to 2+2 = 4 on a pretty consistent basis from a computer. Indeed, we depend on it. Remember, computers are navigating your airliners, keeping you alive in a hospital, sending astronauts to the moon. How can we deal with a computer sometimes saying 2+2 = 2, and taking a different amount of time to do it? You can verify this for yourself – look at the examples when we did training on neural networks. Did we ever achieve a 100% success from a training run, where we got all of the answers right? No, not once. This is because artificial neural Networks are universal approximation functions that map inputs – which can be quite complex – to outputs. They do this by dealing in probabilities and averages, which were developed over time. You can think of an artificial neuron to be a probability engine that says, 45 out of the last 50 times I got this set of inputs, the output was supposed to be true. The odds are it will be true this time. And it sets itself to true. We may have millions of little artificial neurons in our network, each of them making the same sort of calculation. The net result is making a very educated guess about the answer. For most applications of our neural networks, this is OK behavior. We are classifying pictures, and it is OK if a few are wrong. We do a Google search for platypus, and we get one picture out of 100 of Platypus brand tennis shoes. That is OK for a Google search, but what if we were doing something more serious – like recognizing pedestrians in a self- driving car. Is it OK if we misidentify one pedestrian out of 100 and not avoid them? Of course not. That is why, right now, we don’t allow AI system in such critical functions. But people want to use AI in this way – in fact, quite a lot. It would be great to have an AI function that recognized geese in flight and told your airliner how to avoid them. It would be great to have an AI recognize that a patient was misdiagnosed in the hospital and needed immediate attention. But we can’t do that until we come up with processes for dealing with the non-deterministic and thus non-reliable nature of AI. Now today, we deal with non-deterministic elements in automobiles all of the time. They are called drivers. We also know that 94% of car crashes (5) are caused by that human element behind the wheel, which is why we need self-driving cars with a better percentage. How do we deal with human drivers? We require them to be a certain age, which means they have gained experience. They have to pass a test, demonstrating competency in accomplishing tasks. They have to demonstrate compliance with rules and regulations by passing a knowledge test. And they have to get periodically re- ® 179 Chapter 4  Managing AI Projects certified by renewing their license. We also require seat belts and airbags to partially mitigate the risk of the human driver making mistakes by reducing some of the resulting injury. We can apply these types of criteria to AI. We can require a certain amount of training cases. We can test and demonstrate a level of competency. We can predict in advance the level of errors or mistakes, and put measures in place to mitigate that risk. Perhaps we can have two AI systems, one that detects obstacles, and the other that is trained to recognize that the first AI has made a mistake. If we have a 90% chance of the first AI being right, and another 90% of the second AI being right, then we have a 90% + (90% of 10%) = 99% chance of avoidance. I think the key to using AI in safety-critical applications is being able to predict risk in advance, and designing in advance to mitigate either the cause of the risk or the effect. ® 180  Product Management with AI Chapter5: Ideating Products with AI Chapter 6: Identifying Market Opportunities for Products Chapter 7: Infrastructure and Tools for Developing AI Products Chapter 8: AI Product Development and Maintenance Framework Module 2 Chapter 9: Commercializing AI Products with Deep Learning Chapter 10: Impact of AI Transformation on Product Management Chapter 11: Channeling Customization of AI Products