AI Enabled IoT: The Future of Connected Devices PDF

AI Enabled IoT: The Future of Connected Devices Exploring the fundamentals, benefits, and challenges What is AI Enabled IoT? AI Enabled IoT: The Basics AI Enabled IoT is a combination of Artificial Intelligence and Internet of Things, allowing devices to collect and analyze data, learn from...

AI Enabled IoT: The Future of Connected Devices Exploring the fundamentals, benefits, and challenges What is AI Enabled IoT? AI Enabled IoT: The Basics AI Enabled IoT is a combination of Artificial Intelligence and Internet of Things, allowing devices to collect and analyze data, learn from it, and make decisions based on that learning, leading to smarter decision-making and automation. Data Collection and Analysis AI Enabled IoT devices collect and analyze real-time data from sensors and other connected devices, providing valuable insights and allowing for more efficient decision- Key Components of AI Enabled IoT Sensors Sensors are devices that collect data from the environment and send it to cloud computing platforms for processing. They are one of the key components of AI Enabled IoT. Cloud Computing Cloud computing platforms are used to process the data collected by sensors in AI Enabled IoT. They are one of the key components of AI Enabled IoT. Machine Learning Algorithms Machine learning algorithms analyze the data collected by sensors in AI Enabled IoT and make decisions based on that analysis. They are one of the key components of AI Enabled IoT. Applications of AI Enabled IoT using Machine Learning Smart Homes and Buildings Machine Learning can be used in smart homes and buildings to automate and optimize various tasks, such as temperature control, energy usage, and security. Industrial IoT Machine Learning can improve efficiency, productivity, and safety in industrial IoT applications by analyzing data from sensors, predictive maintenance, and autonomous quality control. Healthcare IoT Machine Learning can be used in healthcare IoT to improve diagnosis, treatment, and patient outcomes by analyzing data from wearable devices, medical sensors, and electronic health records. Padlet discussion 1: Give an example application of AI enabled IoT using Machine Learning. https://padlet.com/enpauli/b reakout-link/enZlqb8PWNkz4 3QR-y5WkX40QMLKVXmxG 5 Benefits of AI Enabled IoT Improved Efficiency AI Enabled IoT can improve efficiency through real-time monitoring and automation. This allows companies to reduce downtime, optimize resource usage, and improve overall productivity. Enhanced Customer Experience AI Enabled IoT can enhance customer experience by providing real-time insights and personalized services. This allows companies to better understand their customers and tailor their products and  Data Security and Privacy Ensuring data security and privacy is a major challenge for Machine Learning in IoT. Effective security measures must be put in place to safeguard sensitive data and protect against data breaches. Challenge Computational Resources Machine Learning algorithms require large amounts of s and computational resources, which can be a challenge for IoT devices with limited processing power. Innovative Future of solutions are needed to optimize algorithms and reduce computational requirements. Machine AI Explainability Learning Machine Learning models are often perceived as a black box, with little explanation of how they work. This can be a challenge for IoT applications that require transparency for IoT and accountability. Efforts are underway to develop methods for explaining and interpreting Machine Learning models. Future of Machine Learning for IoT Despite these challenges, the future of Machine Learning for IoT is bright, with the potential to transform industries and improve our lives. Machine Learning can help us  AW S A c a d e m y M a c h i n e L e a r n i n g F o u n d a t i o n s Module 2: Introduction to Machine Learning © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Module overview Sections 1. What is machine learning? 2. Business problems solved with machine learning 3. Machine learning process © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 12 reserved. Module objectives At the end of this module, you should be able to: Recognize how machine learning and deep learning are part of artificial intelligence Describe artificial intelligence and machine learning terminology Identify how machine learning can be used to solve a business problem Describe the machine learning process Identify when to use machine learning instead of traditional software development methods © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 13 reserved. Module 2: Introduction to Machine Learning Section 1: What is machine learning ? © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Artificial intelligence, machine learning, and deep learning Artificial intelligence Machine learning Deep learning © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 15 reserved. Artificial intelligence © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 16 reserved. Machine learning Machine learning is the scientific study of algorithms and statistical models to perform a task using inference instead of instructions. Data Model Prediction Machine learning flow © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 17 reserved. Deep learning © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 18 reserved.  Artificial intelligence Section 1 key Machines performing human takeaways tasks Machine learning Training models to make predictions Deep learning Neural networks Technology and economic advancements have made machine learning more accessible to individuals and 19 organizations© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Module 2: Introduction to Machine Learning Section 2: Business problems solved with machine learning © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Common business use cases Recommended items Spam versus Recommendations Fraud regular email © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 21 reserved. Types of machine learning Supervised Machine Reinforcement learning learning learning Unsupervised learning © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 22 reserved. Supervised learning Learn by identifying patterns in data that is already labeled. Binary Fraud detection [0,1] Image recognition Classification Customer retention Multi Medical diagnostics [0,1,2] Supervised Personalized advertising learning Product sales prediction Regression Weather forecasting Market forecasting Population growth prediction © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 23 reserved.  Unsupervised learning The machine must uncover and create the labels itself. Product recommendations Customer segmentation Clustering Targeted marketing Medical diagnostics Unsupervised learning Visualization Dimensionality Natural language processing reduction Data structure discovery Gene sequencing © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 24 reserved. Reinforcement learning Learning through trial and error. Game AI Reinforcement Self-driving cars learning Robotics + 1 + Customer service routing 1 AWS DeepRacer Best when the desired outcome is known but the exact path to achieving it is not known. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 25  When to use machine learning? Classical programming approach Use machine learning when you have  Large datasets, large number of variables Business logic  Lack of clear procedures to obtain the Task solution Procedures  Existing machine learning expertise  Infrastructure already in place to support ML Machine learning approach  Management support for ML Data Model Task © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 26 reserved.  Machine learning applications Section 2 key affect everyday life takeaways Machine learning can be grouped into – Supervised learning Unsupervised learning Reinforcement learning Most problems are supervised learning © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 27 reserved. Module 2: Introduction to Machine Learning Section 3: Machine learning process © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML pipeline: Business problem Business problem Problem formulation © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 29 reserved. ML pipeline: Data preparation Data handling and cleaning Business problem data Problem data data formulation data Collect and Evaluate Name Country Sex dob label data data Richard Roe UK Male 18/2/1972 Paulo Santos Male 11/2/1969 Mrs. Mary Major Denver F 37 Desai, Arnav USA M 2/22/1962 © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 30 reserved. ML pipeline: Iterative model training Business problem Problem Tune model formulation Meets Collect and Evaluate Feature Select and Evaluate busine label data data engineering train model model ss goal? No Feature augmentation Data augmentation © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 31 reserved. ML pipeline: Feature engineering Name Country Sex dob Richard Roe UK Male 18/2/1972 Paulo Santos Male 11/2/1969 ? Mrs. Mary Major Denver F 37 Desai, Arnav USA M 2/22/1962 Name USA UK sex age bm dow target Richard Roe 0 1 0 49 2 5 140,000 Paulo Santos 1 0 0 51 11 7 78,000 Mary Major 1 0 1 37 NAN 0 167,000 Arnav Desai 1 0 0 58 2 4 100,000 © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 32 reserved. ML pipeline: Model training Name USA UK sex age bm dow target 10–20% Richard Roe 0 1 0 49 2 5 140,000 Test data … … … … … … … … 80% Trained Algorithm model {hyperparameters} © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 33 reserved. ML pipeline: Evaluating and tuning the model Name USA UK sex age bm dow target 10-20% Richard Roe 0 1 0 49 2 5 140,000 Test data … … … … … … … … Change 80% predict features Trained Hosted Algorithm model model {hyperparameters} Change hyperparameters © 2020, Amazon Web Services, Inc. or its Affiliates. All rights Metrics 34 reserved. Overfitting and underfitting Y Y Y X X X Overfitting Underfitting Balanced © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 35 reserved. ML pipeline: Deployment New data, retraining Deploy model Business problem Problem Yes Tune model formulation Meets Collect and Evaluate Feature Select and Evaluate busine label data data engineering train model model ss goal? No Feature augmentation Data augmentation © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 36 reserved.  Machine learning pipeline Section 3 key guides you through the takeaways process of evaluating and training a model Iterative process of – Data processing Training Evaluation © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 37 reserved.  AW S A c a d e m y M a c h i n e L e a r n i n g F o u n d a t i o n s Module 3: Implementing a Machine Learning Pipeline © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Module overview Sections 1. Scenario introduction 2. Collecting and securing data 3. Evaluating your data 4. Preprocessing your data 5. Training 6. Evaluating the accuracy of the model 7. Hosting and using the model © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 39 reserved. Module objectives At the end of this module, you should be able to: Formulate a problem from a business request Obtain and secure data for machine learning (ML) Outline the process for evaluating data Explain why data needs to be preprocessed Use open-source tools to examine and preprocess data How to train and host an ML model Use cross-validation to test the performance of an ML model Use a hosted model for inference © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 40 reserved. Module 3: Implementing a Machine Learning Pipeline Section 1: Scenario introduction © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.  Machine learning pipeline Section 6 New data and retraining Deploy Business model problem Section Section 1 8 Problem Yes Tune model formulation Section Section Section Section Section 2 3 4 5 7 Meets Collect and Evaluate Feature Select and Evaluate busines label data data engineering train model model s goal? No Feature augmentation Data augmentation © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 42 reserved. Machine learning pipeline New data and retraining Deploy Business model problem Problem Yes Tune model formulation Meets Collect and Evaluate Feature Select and Evaluate busines label data data engineering train model model s goal? No Feature augmentation Data augmentation © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 43 reserved. Define business objective What is the business goal? Questions to ask: How is this task done today? How will the business measure success? How will the solution be used? Do similar solutions exist, which you might learn from? What assumptions have been made? Who are the domain experts? © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 44 reserved. How should you frame this problem? Is the problem a machine learning problem? Is the problem supervised or unsupervised? Supervised What is the target to predict? learning Do you have access to the data? What is the minimum performance? Unsupervised Reinforcement learning learning How would you solve this problem manually? What’s the simplest solution? © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 45 reserved. Example: Problem formulation You want to identify fraudulent credit card transactions so that you can stop the transaction before it processes. Why? Reduce the number of customers who end their membership because of fraud. Can you measure it? 10% reduction in fraud Move from qualitative statements to quantitative statements that can be measured. claims in retail © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 46 reserved. Make it an ML model Credit card transaction is either fraudulent or not fraudulent. Fraud Binary classification problem Not fraud Use historical data of fraud reports to help define your model. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 47 reserved. Padlet discussion 2: Formulate a machine learning problem for AI- enabled IoT. https://padlet.com/enpauli/ breakout-link/xPdL4geP6Na VqpwK-y5WkX40QMLKVXm xG 48  Business problems must be Section 1 key converted to an ML problem takeaways Why? Can it be measured? What kind of ML problem is it? Classification or regression? © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 49 reserved. Module 3: Implementing a Machine Learning Pipeline Section 2: Collecting and securing data © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Machine learning pipeline New data and retraining Deploy Business model problem Problem Yes Tune model formulation Meets Collect and Evaluate Feature Select and Evaluate busines label data data engineering train model model s goal? No Feature augmentation Data augmentation © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 51 reserved. What data do you need? How much data do you have, and where is it? Do you have access to that data? What solution can you use to bring all of this data into one centralized repository? © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 52 reserved. Data sources Private data: Data that customers create Commercial data: AWS Data Exchange, AWS Marketplace, and other external providers Open-source data: Data that is publicly available (check for limits on usage) Kaggle World Health Organization U.S. Census Bureau National Oceanic and Atmospheric Administration (U.S.) UC Irvine Machine Learning Repository AWS © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 53 reserved. Observations ML problems need a lot of data—also called observations— where the target answer or prediction is already known. Customer Date of Vendor Charge Was this transactio amount fraud? Feature n ABC 10/5 Store 1 10.99 No DEF 10/5 Store 2 99.99 Yes Target GHI 10/5 Store 2 15.00 No JKL 10/6 Store 2 99.99 ? MNO 10/6 Store 1 99.99 Yes © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 54 reserved. Get a domain expert Do you have the data that you need to try to address this problem? Is your data representative? © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 55 reserved.  The first step in the machine Section 2 key learning pipeline is to obtain takeaways data Extract, transform, and load (ETL) is a common term for obtaining data for machine learning Securing your data includes both controlling access and encrypting data © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 56 reserved. Module 3: Implementing a Machine Learning Pipeline Section 3: Evaluating your data © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Machine learning pipeline New data and retraining Deploy model Business problem Problem Format data Yes Tune model formulation Examine data types Collect and Evaluate Perform descriptive Feature Select and statistics Evaluate Meets busine label data data engineering train model model ss Visualize data goal? No Feature augmentation Data augmentation © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 58 reserved. You must understand your data Before you can run statistics on your data, you must ensure that it’s in the right format for analysis. Date of Charge Was This Customer Transactio Vendor Amount Fraud? n “Customer:ABC,Date ABC 10/5 Store 1 10.99 No OfTransation:10/5,Ven DEF 10/5 Store 2 99.99 Yes dor:Store1,ChargeAm ount:10.99,WasThisFr GHI 10/5 Store 2 15.00 No uad:No…” JKL 10/6 Store 2 99.99 ? MNO 10/6 Store 1 99.99 Yes © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 59 reserved. Loading data into pandas Reformats data into tabular representation (DataFrame) Converts common formats like comma-separated values (CSV), JavaScript Object Notation (JSON), Excel, Pickle, and others import pandas as pd url = "https://somewhere.com/winequality-red.csv" df_wine = pd.read_csv(url,';') © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 60 reserved. pandas DataFrame Number of instances df_wine.shape Number of attributes df_wine.head(5) Columns/Attributes Rows/Instances © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 61 reserved. Index and column names df_wine.columns Index(['fixed acidity', 'volatile acidity', 'citric acid', 'residual sugar', 'chlorides', 'free sulfur dioxide', 'total sulfur dioxide', 'density', 'pH', 'sulphates', 'alcohol', 'quality'], dtype='object') df_wine.index RangeIndex(start=0, stop=1599, step=1) © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 62 reserved.  DataFrame schema df_wine.dtypes df_wine.info() quality int64 fixed acidity float64 Int64Index: 1597 entries, volatile acidity float64 0 to 1598 citric acid float64 Data columns (total 12 columns): residual sugar float64 quality 1597 non-null int64 chlorides float64 fixed acidity 1597 non-null float64 free sulfur dioxide float64 volatile acidity 1597 non-null float64 total sulfur dioxide float64 citric acid 1597 non-null float64 density float64 residual sugar 1597 non-null float64 pH float64 chlorides 1597 non-null float64 sulphates float64 free sulfur dioxide 1597 non-null float64 alcohol float64 total sulfur dioxide 1597 non-null float64 dtype: object density 1597 non-null float64 pH 1597 non-null float64 sulphates 1597 non-null float64 alcohol 1597 non-null float64 dtypes: float64(11), int64(1) © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. memory usage: 162.2 KB 63 Descriptive statistics Use descriptive statistics to gain insights into your data before you clean the data: Overall statistics Multivariate statistics Attribute statistics © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 64 reserved.  Statistical characteristics df_wine.describe() free total fixed volatile citric residua chlorid sulpha sulfur sulfur pH alcohol quality acidity acidity acid l sugar es tes dioxide dioxide count 1599.00 1599.00 1599.00 1599.00 1599.00 1599.00 1599.00 1599.00 1599.00 1599.00 1599.00 mean 8.32 0.53 0.27 2.54 0.09 15.87 46.47 3.31 0.66 10.42 5.64 std 1.74 0.18 0.19 1.41 0.05 10.46 32.90 0.15 0.17 1.07 0.81 min 4.60 0.12 0.00 0.90 0.01 1.00 6.00 2.74 0.33 8.40 3.00 25% 7.10 0.39 0.09 1.90 0.07 7.00 22.00 3.21 0.55 9.50 5.00 50% 7.90 0.52 0.26 2.20 0.08 14.00 38.00 3.31 0.62 10.20 6.00 75% 9.20 0.64 0.42 2.60 0.09 21.00 62.00 3.40 0.73 11.10 6.00 max 15.90 1.58 1.00 15.50 0.61 72.00 289.00 4.01 2.00 14.90 8.00 © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 65 reserved. Categorical statistics identify frequency of values and class imbalances df_car.head(5) person lug_bo buying maint doors safety class s ot 0 vhigh vhigh 2 2 small low unacc 1 vhigh vhigh 2 2 small med unacc 2 vhigh vhigh 2 2 small high unacc 3 vhigh vhigh 2 2 med low unacc 4 vhigh vhigh 2 2 med med unacc df_car.describe() person lug_bo buying maint doors safety class s ot count 1728 1728 1728 1728 1728 1728 1728 unique 4 4 4 3 3 3 4 top low low 2 2 big low unacc freq 432 432 432 576 576 576 1210 © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 66 reserved. Plotting attribute statistics df_wine[‘sulphates’].hist(bins=10) df_wine[‘sulphates’].plot.box() df_wine[‘sulphates'].plot.kde() © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 67 reserved. Plotting multivariate statistics df_wine.plot.scatter( pd.plotting.scatter_matrix( x='alcohol', df_wine[['citric acid', y='sulphates') 'alcohol', 'sulphates']]) © 2020, Amazon Web Services, Inc. or its Affiliates. All rights 68 reserved.  Scatter plot with identification high = df_wine[['sulphates','alcohol']][df_wine['quality']>5] low = df_wine[['sulphates','alcohol']][df_wine['quality']