Fundamentals of AI and Machine Learning PDF

Fundamentals of AI and Machine Learning Overview of Machine Learning and Its Importance  Traditional Machine Learning Approach Imagine developing an accurate audio recognition system that distinguishes between laughing, shouting, and speaking, for customer se...

Fundamentals of AI and Machine Learning Overview of Machine Learning and Its Importance  Traditional Machine Learning Approach Imagine developing an accurate audio recognition system that distinguishes between laughing, shouting, and speaking, for customer service automation, safety alerts, and interactive learning tools. Traditional approach involves: Training the Testing and Writing code Collecting data model validating This method is effective, but it is very time-consuming and requires advanced coding and machine learning skills.  Simplified No-Code Machine Learning Approach Now, picture the same project scenario with the approach of Teachable Machine, a user-friendly tool that requires no coding skills. https://teachablemachine.withgoogle.com/  Learning Objectives By the end of this lesson, you will be able to: Identify key elements of machine learning to understand its application in various real-world scenarios Differentiate between AI, ML, and DL to recognize their distinct roles and applications in technology Identify the stages of the ML lifecycle to develop and implement machine learning models effectively Explain the importance of MLOps to grasp how it enhances the efficiency and scalability of ML models Machine Learning (ML)  What Is Machine Learning? Machine learning is a subset of artificial intelligence that enables computers to learn from and make predictions based on data without being explicitly programmed. Machine learning systems, unlike traditional programming, learn from identifying patterns and making inferences from data. Example: Spotify employs a machine learning system to suggest new songs based on past listening preferences.  How Does Machine Learning Work? The image below illustrates the machine learning process. Input data Analyze data Find pattern Prediction Decision-making  Importance of Machine Learning Machine learning has become increasingly important in today's digital age due to several factors: Decision-making Data explosion support Innovation and Competitive Adaptability and automation advantage scalability  Difference Between AI, ML, and DL AI is a broader field that includes machine learning, which in turn includes deep learning. This hierarchy implies that DL is a specialized area within ML, which itself falls under the broader scope of AI. Artificial Machine Deep Intelligence (AI) Learning (ML) Learning (DL)  Artificial Intelligence (AI) AI is the simulation of human intelligence processes by machines, enabling them to perform tasks that typically require human intelligence. Example: A self-driving car It can navigate, make decisions, and avoid obstacles much like a human driver would. The entire system, including sensors, software, and decision-making algorithms, is part of the AI that enables the car to drive autonomously.  Machine Learning (ML) ML is a subset of AI where algorithms are trained on data to make predictions or decisions without being explicitly programmed. Example: A self-driving car Machine learning algorithms in self-driving cars identify objects such as pedestrians, vehicles, and traffic signs. A car's vision system uses a machine learning model trained on numerous images of roads and traffic conditions to identify and respond to different scenarios.  Deep Learning (DL) DL is a subset of ML that uses multi-layered neural networks to analyze complex data and identify patterns for advanced decision-making. Example: A self-driving car In self-driving cars, deep learning assists in real-time image and video analysis. The car employs a convolutional neural network (CNN) to analyze video feeds for detecting lane lines, recognizing traffic lights, and predicting movements of people and other vehicles.  Application Domains of Machine Learning Healthcare Finance Disease diagnosis and prognosis Fraud detection and prevention Medical image analysis such as MRI and CT scans Credit scoring and risk assessment Drug discovery and development Algorithmic trading and stock market prediction Dietary recommendations Customer segmentation and targeting Health monitoring and wearable devices Portfolio management and asset allocation  Application Domains of Machine Learning Retail and e-commerce Marketing and advertising Product recommendation systems Targeted advertising Customer churn prediction and retention Customer segmentation and profiling Demand forecasting and inventory management Customer churn prediction Price optimization and dynamic pricing Social media analytics and sentiment analysis Customer sentiment analysis and feedback Content recommendation and personalization analysis  Application Domains of Machine Learning Manufacturing and industry Transportation and logistics Route optimization and vehicle routing Predictive maintenance of machinery and equipment Predictive maintenance for fleets and vehicles Quality control and defect detection Demand forecasting for ride-sharing and delivery services Supply chain optimization and logistics Traffic flow prediction and congestion Process optimization and efficiency improvement management Predictive analytics for production planning Autonomous navigation for vehicles and drones Types of Machine Learning  Types of Machine Learning Machine learning is categorized into four main approaches, each designed to address different kinds of data and learning tasks. Semi- Supervised Unsupervised Reinforcement supervised learning learning learning learning  Supervised Learning Supervised learning involves training a model on labeled data to predict outputs for new, unseen inputs. The model identifies patterns in the data during training and uses these patterns to make predictions. It includes two types: classification, which predicts categorical labels, and regression, which estimates continuous numerical values.  Supervised Learning: Example Consider a model that uses labeled input data to predict the category of new, unlabeled data. Input data: Includes images of apples that the model uses for learning Annotations: Provide labels stating "These are apples" to teach the model what the image represents Model: Learns to recognize apples by analyzing the input data and annotations Prediction: Applies its learning to identify a new image as an apple  Unsupervised Learning Unsupervised learning analyzes datasets to discover underlying patterns and structures without the need for labeled data. It autonomously finds relationships in data, categorizing information based on similarities and differences without prior human input. It includes two types: clustering, which groups data based on similarities, and association, which identifies common patterns.  Unsupervised Learning: Example Consider a scenario where a model processes raw input data to identify patterns and group similar data into categories without prior labels. Input data: Includes images of apples and bananas to be analyzed by the model Output Model: Learns to differentiate and group similar items autonomously Output: Classifies and separates apples and bananas based on similarities  Semi-Supervised Learning Semi-supervised learning falls between supervised and unsupervised learning. It uses labeled data for guidance and unlabeled data to enhance model performance. It's beneficial when labeled data is limited or costly, making effective Hexagon Square use of both data types for better accuracy and adaptability. Triangle  Semi-supervised Learning: Example Consider a scenario where the input data includes both labeled and unlabeled data. Input data: Consists of labeled and unlabeled data Model: Learns to classify fruits by analyzing the labeled data Prediction: Classifies the unlabeled image as an apple using past and current learned patterns  Reinforcement Learning Reinforcement learning involves an agent making decisions by performing actions and receiving feedback in the form of rewards or penalties. The agent seeks to maximize rewards over time by experimenting with different strategies and learning from the outcomes of its actions. The agent continuously improves its decision-making process through trial and error to achieve the best possible results.  Reinforcement Learning: Example Consider an example where a robot learns to navigate its environment through reinforcement learning. ?? Stage 1 Stage 1 Ouch! Environment and Agent: The robot is in a setting Stage 2 -50 points with a fire, a water faucet, and a bucket. Exploration: The robot is uncertain about which actions are beneficial or harmful and begins = bad exploring its surroundings. Next time avoid it. Stage 3  Reinforcement Learning: Example Consider an example where a robot learns to navigate its environment through reinforcement learning. ?? Stage 1 Stage 2 Ouch! Action: The robot approaches the fire. Stage 2 -50 points Feedback: The robot gets burned and incurs a penalty of -50 points, learning that this action is undesirable. = bad Next time avoid it. Stage 3  Reinforcement Learning: Example Consider an example where a robot learns to navigate its environment through reinforcement learning. ?? Stage 1 Stage 3 Ouch! Realization: The robot processes the feedback Stage 2 -50 points and recognizes that fire is dangerous. Adaptation: It adjusts its behavior to avoid the fire in the future, thus improving its decision- = bad making process. Next time avoid it. Stage 3 Quick Check A retail company collects extensive data on how customers browse products on their website but does not have specific labels or categories assigned to these behaviors. They wish to understand if there are any natural groupings of similar browsing habits. Which type of machine learning should be used for this task? A. Supervised learning B. Unsupervised learning C. Semi-supervised learning D. Reinforcement learning Traditional Programming vs. Machine Learning Approach  Traditional Approach: Rule-Based Systems A rule-based system is a computing system that makes decisions by following a set of predefined rules. These rules are like instructions or guidelines that tell the system what to do when certain conditions are met.  Rule-Based Systems: Example Consider spam filtering for emails that uses predefined rules to classify incoming emails as spam or not. For instance, the rules might include: Rule 1: Classify emails with keywords such as "free," "discount," or "promotion" as spam. Rule 2: Classify emails from unknown or suspicious senders as spam. Rule 3: Classify emails with many links or attachments as spam. Rule 4: Classify emails flagged as spam by other users or sent to many recipients as spam. By applying these rules, the system automatically sorts spam emails, keeping the inbox clean.  Machine Learning Approach: Data-Driven Systems Data-driven systems use algorithms to analyze large amounts of data, identify patterns, and make informed decisions or predictions. This approach excels in handling complex, nuanced problems where defining explicit rules is not possible.  Data-Driven Systems: Example Consider spam filtering for emails that uses data-driven algorithms to analyze patterns and classify incoming emails as spam or not. The steps include: Data collection: Gather email content, sender details, and user spam markings. Pattern recognition: Identify spam traits such as suspicious keywords or unfamiliar domains. Email classification: Sort emails instantly, directing spam to a specific folder. Feedback loop: Use user feedback to refine the process and improve accuracy. As a result, this data-driven approach increases the accuracy of spam detection and adapts to evolving threats, ensuring a cleaner inbox. The Paradigm Shift from Rule-Based to Data-Driven Approach The paradigm shift allows: Adaptability and flexibility Improved performance and accuracy Scalability Machine Learning Lifecycle  Predicting Customer Churn A retail company is experiencing a higher-than-acceptable customer churn rate, which is negatively impacting its revenue and long-term customer relationships. The company understands that identifying the key factors influencing customers' decisions to leave is crucial for developing effective retention strategies. Challenge: The extensive variety and amount of data from sales, feedback, online behavior, and loyalty programs hinder manual analysis and insights.  Predicting Customer Churn To overcome the challenge, the company has opted to use machine learning to systematically analyze diverse datasets and identify patterns contributing to customer churn. By adopting the machine learning lifecycle, they can efficiently manage the stages from data collection and preparation to model building.  Machine Learning Lifecycle The various stages in a machine learning lifecycle are: Understanding Data collection the problem 1 2 Monitoring and 8 3 Data preparation maintenance Model 7 4 Exploratory data deployment analysis (EDA) 6 5 Model evaluation Model building  Machine Learning Lifecycle The first stage involves defining the goal that the system is intended to address. Understanding the problem 1 It focuses on identifying the key objectives and It entails understanding the constraints and the desired outcomes. requirements necessary to achieve these goals. Example: A retail company wants to reduce customer churn by identifying which factors are most influential in customer decisions to leave.  Machine Learning Lifecycle The second stage encompasses the gathering of relevant data that will be used to train and test the machine learning model. Understanding the 1 2 Data collection problem It entails choosing the right data sources like customer databases, transaction logs, and online interactions. Example: The retail company collects data from various sources, including sales records, customer service interactions, and online browsing patterns.  Machine Learning Lifecycle The third stage involves cleaning and organizing the collected data to make it suitable for analysis. Understanding the 1 2 Data collection problem 3 Data preparation This includes error removal, handling missing values, and transforming the data into the required format. Example: The collected data is then cleaned of errors and inconsistencies, formatted correctly, and enriched with additional variables that might be useful for analysis.  Machine Learning Lifecycle The fourth stage involves analyzing the data to discover patterns, trends, and relationships, which helps in understanding the data better. Understanding the 1 2 Data collection problem 3 Data preparation 4 Exploratory data analysis (EDA) The analysis includes identifying outliers or Statistical tests are conducted to validate the anomalies that may affect the model's performance. significance of the relationships found in the data. Example: Data scientists perform statistical analyses and visualize the data to uncover trends, patterns, and anomalies that could affect churn, such as purchase frequency or complaints.  Machine Learning Lifecycle The fifth stage involves constructing the machine learning model using insights gained from the exploratory data analysis. Understanding the 1 2 Data collection problem 3 Data preparation 4 Exploratory data analysis (EDA) 5 Model building It includes selecting the appropriate algorithms and techniques based on the identified patterns and relationships in the data. Example: Using the insights from EDA, the team develops a machine learning model using algorithms that can forecast customer churn based on identified patterns.  Machine Learning Lifecycle The sixth stage involves assessing the performance of the model to determine its accuracy and effectiveness in making predictions or decisions. Understanding the 1 2 Data collection problem 3 Data preparation 4 Exploratory data analysis (EDA) 6 5 Model validation Model building The team evaluates the model by testing it with new data to assess its predictive accuracy. Example: The model is tested with new data, such as recent customer interactions and sales figures, to assess its accuracy and effectiveness in predicting customer churn.  Machine Learning Lifecycle The seventh stage involves deploying the model into the company's systems to make predictions. Understanding the 1 2 Data collection problem 3 Data preparation Model 7 4 Exploratory data deployment analysis (EDA) 6 5 Model validation Model building The model is integrated into existing operational workflows, enabling it to analyze data and provide insights or recommendations in real time. Example: Tested with recent customer and sales data, the model is deployed into the retail company's CRM system to offer real-time insights to customer service representatives.  Machine Learning Lifecycle The last stage involves continuously monitoring and updating the model to maintain its performance over time. Understanding the 1 2 Data collection problem Monitoring and 8 3 Data preparation maintenance Model 7 4 Exploratory data deployment analysis (EDA) 6 5 Model validation Model building Regular assessments help identify areas for improvement, allowing for timely adjustments that enhance accuracy and reliability. Example: The model's performance in the retail company is regularly monitored and updated with new data to adapt to evolving customer behavior and market trends. Demo: Image Recognition with Teachable Machine Duration: 20 minutes Objective: To explore the Image Project in Teachable Machine, focusing on developing a facial recognition model that enhances device security and illustrates the machine learning lifecycle. Understanding Generalization, Overfitting, and Underfitting in Machine Learning  Generalization Generalization is the model’s ability to apply what it has learned from the training data to new data it has never seen before. A model that generalizes well can accurately predict outcomes not just on the data it was trained on but also on any new data.  Generalization: Example Consider a weather prediction model trained on historical weather data from various locations over many years. The model processes data on temperature, humidity, pressure, and wind to learn weather relationships. It recognizes patterns such as pressure drops signaling storms and specific conditions causing fog. It accurately predicts weather for new places and times due to its extensive experience with diverse climates.  Overfitting Overfitting occurs when a model learns the training data too well, including its noise and outliers, making it perform exceptionally on the training data but poorly on new, unseen data. This results in a model that is too specific to the training data and lacks the flexibility to adapt to new situations.  Overfitting: Example Consider a weather prediction model that has been trained extensively on a specific set of historical weather data for a particular location and time period. It captures every minor detail, including irrelevant fluctuations like brief temperature spikes, and disturbances from sensor errors or environmental interference. It performs poorly in new places or times as it is specifically tailored to the training data. This results in the model making unreliable predictions when encountering different conditions.  Underfitting Underfitting happens when a model is too simple to capture the underlying patterns in the training data, resulting in poor performance. This occurs as the model does not learn sufficiently from the training data, resulting in inaccuracies both in the training set and with new, unseen data.  Underfitting: Example Consider a weather prediction model trained on only a small set of data, such as average temperatures and wind patterns. It might only use average temperature, ignoring other important factors like atmospheric pressure and precipitation. The model fails to grasp weather complexity, leading to inaccurate predictions. When evaluated on new data, its predictions are inaccurate due to insufficient learning about weather factors. Quick Check A team developed a recipe recommendation app that works perfectly in tests with their own sample recipes but fails to provide good suggestions when presented with new recipes. What is likely the problem with their app? A. Generalization B. Underfitting C. Overfitting D. None of the above Introduction to Machine Learning Operations (MLOps)  MLOps MLOps is a set of practices and tools that streamline the deployment and management of machine learning models in production environments. It combines aspects of machine learning and operations to improve the efficiency, reliability, and scalability of ML systems. MLOps ensures efficient deployment, management, and updating of machine learning models, helping organizations derive value from data-driven applications.  MLOps Practices MLOps practices are methods and tools that streamline the lifecycle of machine learning models, from development through deployment and maintenance. Key practices include: Continuous integration and Model versioning continuous deployment Model monitoring (CI/CD) Experiment tracking Automated data validation  Benefits of MLOps MLOps offers numerous benefits making it a key driver of success in machine learning projects. Some of the benefits are: Improved Faster model deployment performance Cost Better risk Scalability efficiency management  Examples of MLOps Some of the real-world examples of successful MLOps implementations are: Spotify uses MLOps practices to manage its recommendation systems, continuously updating and personalizing playlists and recommendations for users. Walmart uses MLOps to optimize machine learning models in over 10,000 stores across 24 countries, improving operational efficiency and customer satisfaction through data-driven decisions. Netflix uses AI to enhance its expanding catalog of movies and TV shows. It uses real-time model monitoring through MLOps to adjust recommendations based on viewer behavior predictions. Quick Check GlobalMart, a multinational retailer, has implemented machine learning operations (MLOps) to better manage their inventory and enhance customer satisfaction across their global stores. What is the primary benefit of using MLOps in this context? A. Reducing employee training times B. Enhancing the efficiency of inventory management C. Improving the aesthetics of their website D. Increasing the variety of products offered  Guided Practice Overview Duration: 30 minutes This activity involves a detailed exploration of a machine learning application through a case study on Glovo's use of the Kustomer platform to enhance customer service. It showcases how AI can streamline operations, improve efficiency, and boost customer satisfaction in a multilingual, high-volume environment. You are prompted to analyze AI techniques, understand decision-making processes for technology adoption, and identify direct business impacts.  Key Takeaways By now, you have learned: Machine learning is a subset of AI that enables computers to learn from data and make predictions or decisions without explicit programming. Machine learning can be classified into supervised, unsupervised, semi-supervised, and reinforcement learning. The machine learning lifecycle includes crucial stages such as data collection, preparation, model building, evaluation, and deployment, which are necessary to develop and implement ML models effectively. MLOps streamlines the deployment, monitoring, and maintenance of machine learning models in production, ensuring they operate efficiently and adapt to new data and environments in real time. Q&A