Introduction to AI and Deep Learning PDF

Introduction to AI and Deep Learning Jaeil Kim, Ph.D. School of Computer Science and Engineering Kyungpook National University  01 Tutorial Overview 02 AI and ML Index 03 PyTorch and TensorFlow 04 Setting Up Dev Enviro...

Introduction to AI and Deep Learning Jaeil Kim, Ph.D. School of Computer Science and Engineering Kyungpook National University  01 Tutorial Overview 02 AI and ML Index 03 PyTorch and TensorFlow 04 Setting Up Dev Environment 0 Lecturer: Jaeil Kim, Ph.D. Computer-aided diagnosis - Breast cancer diagnosis using US and PET-CT - Chest disease diagnosis using plain radiographs - Brain disease diagnosis using cortical surface - Brain hemorrhage detection using CT Tasks - Image classification - Weakly-supervised localization - Graph learning and classification - Image segmentation - Anomaly detection (normality learning) Jaeil Kim, KNU ([email protected]) 3 0 Lecturer: Jaeil Kim, Ph.D. Coastal Video Analysis & Climate Prediction - Coastal wave tracking and analysis using CCTV - Wave overtopping detection - Wave parameter prediction - El Niño & La Niña prediction Tasks - Image distortion removal - Video tracking/image registration - Time-series prediction Jaeil Kim, KNU ([email protected]) 4 0 Lecturer: Jaeil Kim, Ph.D. Intelligent Inspection System - Malware traffic detection - Non-destructive inspection equipment - Abnormal pattern recognition for fabric production Tasks - Anomaly detection (normality learning) - Time-series classification - Unsupervised learning Jaeil Kim, KNU ([email protected]) 5 0 Course Introduction This course introduces basic theories and various algorithms for machine learning and deep learning. ◦ Students will learn a variety of supervised and unsupervised learning techniques with deep learning to solve practical problems. ◦ Classes on theoretical and algorithmic aspects are complemented by simple projects and homework. Grade ◦ Midterm 35% ◦ Final 35% ◦ Presentation 10% ◦ Project 15% ◦ Challenge 5% ◦ From A+ to C0 ✔ No exceptions. If you need a good grade for your scholarship, you must submit every homework and get good marks in the exams. ✔ Plagiarism in homework, missing an exam → F Jaeil Kim, KNU ([email protected]) 6  2 Artificial Intelligence Baker, Bowen, et al. "Emergent tool use from multi-agent autocurricula." arXiv preprint arXiv:1909.07528 (2019). Jaeil Kim, KNU ([email protected]) 7  2 Artificial Intelligence Designing rational agents ◦ Agent: anything that perceives its environment through sensors and acts through actuators ✔ Human, robot, software, … ◦ Percepts: agent’s perceptual inputs ✔ Percept sequence: history of everything that the agent perceived ◦ Agent function: mapping percept sequence to an action https://www.samsung.com/us/business/home-appliances/vacuums/jet- Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Novig bot-robot/jet-bot-plus-robot-vacuum-with-clean-station-vr30t85513w-aa/ Jaeil Kim, KNU ([email protected]) 8 2 Machine Learning “A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E” ◦ Mitchell, T. (1997). Machine Learning. McGraw Hill. p. 2. Classify the black and pink dots - Using dataset with three classes 1. Which one is easier? 2. What’s your strategy to solve this problem? Jaeil Kim, KNU ([email protected]) 9 2 Machine Learning Machine learning solution Dataset/Experience Model 2D Vector for each point 𝑀 𝑥𝑖 Distance to class center Single Value for classes = 𝐶(𝑑0 𝑥𝑖 , 𝑑1 𝑥𝑖 , 𝑑2 𝑥𝑖 ) Machine Learning Optimization Task Finding class center 1, 𝑖𝑓 𝑀 𝑥𝑖 == 𝑐𝑖 Optimizing parameters 0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 Jaeil Kim, KNU ([email protected]) 10  2 Development Phases Task? Experience? Performance Measure? Model? Data Data Pre- Cleansing processing Problem Data Data Model Model Definition Collection Labeling Training Deployment Data Model Analysis Validation Data Preparation Learning Phase Deployment Phase Jaeil Kim, KNU ([email protected]) 11 2 Machine Learning Breast cancer diagnosis using US Task: classifying US images into normal, benign and malignant Experience: ultrasound images + labels (0/1 or more) Performance: number of instances correctly classified Model: input & output, model definition * Program to be updated (optimized) Jaeil Kim, KNU ([email protected]) 12 2 Machine Learning Time-series prediction Task? Experience? Performance Measure? Model? Jaeil Kim, KNU ([email protected]) 13 2 Machine Learning Phases Jaeil Kim, KNU ([email protected]) 14 2 Machine Learning Tasks and performance measure ◦ Making inferences from a sample Types of machine learning algorithms ◦ Learning associations ◦ Supervised learning ✔ Classification, regression ◦ Unsupervised learning ◦ Reinforcement learning Source: https://towardsdatascience.com/self- learning-ai-agents-part-i-markov-decision- processes-baf6b8fc4c5f Jaeil Kim, KNU ([email protected]) 15 2 Machine Learning Learning associations ◦ 𝑋→𝑌 ◦ People who buy/click/visit/enjoy X are also likely to buy/click/visit/enjoy Y ◦ A rule implies association, not necessarily causation Basket analysis ◦ P (Y | X ) probability that somebody who buys X also buys Y where X and Y are products/services. ◦ Support (𝑋 → 𝑌) # customers who bought X and Y  P( X ,Y ) = # customers  ◦ Confidence (𝑋 → 𝑌) P ( X ,Y ) P (Y | X ) = P( X ) # customers who bought X and Y  = # customers who bought X Jaeil Kim, KNU ([email protected]) 16 2 Machine Learning Supervised learning ◦ Teach the computer how to do something, then let it use its knowledge to do it 𝑛−1 1 2 ෍ 𝑦𝑖 − 𝑀(𝑥𝑖 ) 𝑛 𝑖=0 Applications ◦ Prediction of future cases: Use the rule to predict the output for future inputs ◦ Knowledge extraction: The rule is easy to understand ◦ Compression: The rule is simpler than the data it explains Classification and regression Jaeil Kim, KNU ([email protected]) 17 2 Machine Learning Classification ◦ A process related to categorization, the process in which ideas and objects are recognized, differentiated and understood *Wikipedia Jaeil Kim, KNU ([email protected]) 18 2 Machine Learning Regression ◦ “Statistical processes for estimating the relationships among variables.” ◦ Modeling the relationships → prediction of a variable from other variables (x,y) α1= g1(x,y) α2= g2(x,y) α2 α1 https://www.alanzucconi.com/2017/04/10/robotic-arms/ Jaeil Kim, KNU ([email protected]) 19 2 Machine Learning Unsupervised learning ◦ Let the computer learn how to do something, and use the model to determine structure and patterns in data ◦ Using unlabeled data Learning “what normally happens” ◦ Clustering: Grouping similar instances ◦ Dimension reduction ◦ Learning normality https://medium.com/@b.terryjack/unsuper vised-learning-to-aid-labelling-for- supervised-learning-253fe2d8e06b https://www.researchgate.net/figure/Examples-of-Supervised-Learning-Linear-Regression-and-Unsupervised- Learning_fig3_336642133 Jaeil Kim, KNU ([email protected]) 20 2 Machine Learning Reinforcement learning ◦ How the computer ought to take actions in an environment so as to maximize some notion of cumulative reward ◦ Learning a policy: A sequence of outputs ◦ No supervised output but delayed reward https://developer.nvidia.com/blog/nvidia-sponsors- learning-to-run-ai-competition-at-nips-2017/ Tang, Xianfeng, et al. "Joint modeling of dense and incomplete trajectories for citywide traffic volume inference." The World Wide Web Conference. 2019. Jaeil Kim, KNU ([email protected]) 21 2 AI & ML Jaeil Kim, KNU ([email protected]) 22 2 Deep Learning “Get a Program from Data” Hosny, Ahmed, et al. "Artificial intelligence in radiology." Nature Reviews Cancer 18.8 (2018): 500-510. Jaeil Kim, KNU ([email protected]) 23 2 Deep Learning Early diagnosis of Alzheimer’s disease using medical imaging ◦ AD: irreversible disease  early detection is very important for treatment Using deep learning ◦ 75.8 months earlier than medical doctors ◦ Machine can learn very complex patterns what humans cannot observe Jaeil Kim, KNU ([email protected]) 24 3 Deep Learning Frameworks https://www.kaggle.com/getting-started/156399 Jaeil Kim, KNU ([email protected]) 25 3 Deep Learning Frameworks https://www.kaggle.com/getting-started/156399 Jaeil Kim, KNU ([email protected]) 26 3 Deep Learning Frameworks https://developer.ibm.com/blogs/announcing-pytorch-1-support-in-fabric-for-deep-learning/ Jaeil Kim, KNU ([email protected]) 27