AI216 Machine Learning and Pattern Recognition - Delta University PDF

AI216 Machine Learning and Pattern Recognition Delta University for Sciences and Technology Faculty of Artificial Intelligence Fall 2024/2025  Lectures Day Group Time Hall...

AI216 Machine Learning and Pattern Recognition Delta University for Sciences and Technology Faculty of Artificial Intelligence Fall 2024/2025  Lectures Day Group Time Hall Activity Sunday (B) 8:45 – 10:15 (438) Hall 8 Class (A) 10:15 – 11:45 (435) Hall 7 Monday (D) 8:45 – 10:15 (438) Hall 8 (C) 10:15 – 11:45 (435) Hall 7 TBD ALL Groups TBD My office Office Hour ❑ Other times are available by appointment ❑ The best way to contact me is by email Machine learning and Credit Title: Code: AI216 Pattern Recognition hours:3 Introduction to Artificial Credit Perquisite: Code: AI127 Intelligence hours:3 4 Lecture: 2 Tutorial: - Practical: 2 Total: hour/week Dr. Hanaa Salem Marie Fall 2024/2025  List of References: No. Reference Type Aurélien Géron, "Hands-On Machine Learning with Scikit-Learn, Keras, and 1. TensorFlow", 2th edition, O’Reilly, September 2019. Textbook John D. Kelleher, Brian Mac Namee, Aoife D'Arcy, “Fundamentals of Machine 2. Learning for Predictive Data Analytics: Algorithms, Worked Examples, and Textbook Case Studies. The MIT Press, July 24th, 2015 Andreas C. Miller & Sarah Guido, “Introduction to Machine Learning with 3. Python, 1st edition, O’Reilly, October 2016. Textbook Electronic 4. http://ciml.info Textbook https://www.datacamp.com/users/sign_up?redirect=%2Fcourses%2Funderst 5. anding-machine-learning%2Fcontinue https://alison.com/tag/machine- Free Online Courses learning?utm_source=google&utm_medium=cpc&utm_campaign=Performa 6. nce-Max_Tier-5_Alison- Profiles&gad_source=1&gclid=Cj0KCQjwr9m3BhDHARIsANut04aqlRuFECg n21ZVVfenEDv0c6uUe2Ux5WXSnJ4g3a5bJfoTD5SgKTUaAsElEALw_wcB Dr. Hanaa Salem Marie Fall 2024/2025  Lecture 2 Introduction to Machine Learning  Roadmap Al, ML, and DL? What is Learning? What is Machine Learning? Steps in machine learning. Types of machine Learning. Applications of Machine Learning.  Artificial Intelligence (Al) Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  Machine Learning (ML) Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  Deep Learning (DL) Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Deep Learning Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  ML & DL Assoc. Prof. Hanaa Salem Marie Spring 2023/2024 What is Machine Learning(ML) ?  What is Machine Learning (ML)? Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  What is Machine Learning (ML)? “Learning is any process by which a system improves performance from experience.” ❑ Herbert Simon Definition by Tom Mitchell (1998): Machine Learning is the study of algorithms that Improve their performance P At some tasks T With experience E A well-defined learning task is given by. Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  What is Machine Learning (ML)? Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  When Do We Use Machine Learning? Assoc. Prof. Hanaa Salem Marie Fall 2024/2025 Some more examples of tasks that are best solved by using a learning algorithm ❑ Recognizing patterns: ▪ Facial identities or facial expressions ▪ Handwritten or spoken words ▪ Medical images ❑ Generating patterns: ▪ Generating images or motion sequences ❑ Recognizing anomalies: ▪ Unusual credit card transactions ▪ Unusual patterns of sensor readings in a nuclear power plant ❑ Prediction: ▪ Future stock prices or currency exchange rates Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  Sample Applications ❑ Web search ❑ Computational biology ❑ Finance ❑ E-commerce ❑ Space exploration ❑ Robotics ❑ Information Extraction ❑ Social networks ❑ Debugging software ❑ [Your favorite area] Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  Let's start with Fruit Classification Fruit Classification Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  Let's start with Fruit Classification Traditional programming Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  Let's start with Fruit Classification Messy Real World Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  Let's start with Fruit Classification Manual — Rules Solutions (1) (2) Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  Problem of Traditional Programming OK — What if a new problem You need to change the rules Traditional programming is Not suitable Machine Learning We need an algorithm that figures out the rules for us So, we don’t have to write them by hand Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  Machine Learning Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  Machine Learning: Classifier Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  Machine Learning: Learning Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  Machine Learning: Learning Data ❑ Training set - Used to fit (train) model parameters ❑ Validation set - Used to check performance on independent data and tune parameters ❑ Test set - Final evaluation of performance after all parameters fixed Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  Machine Learning: Learning Types Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  Machine Learning: Learning Types ❑ Supervised (inductive) learning ✓ Given: training data + desired outputs (labels) ❑ Unsupervised learning ✓ Given: training data (without desired outputs) ❑ Semi-supervised learning ✓ Given: training data + a few desired outputs ❑ Reinforcement learning ✓ Rewards from the sequence of actions Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Machine Learning: Supervised Learning ❑ Supervised Learning: A Technique that builds the classifier Automatically ▪ Create a classifier by finding patterns in the image Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Machine Learning: Supervised Learning Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Machine Learning: Unsupervised Learning Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Machine Learning: Unsupervised Learning Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Machine Learning: Reinforcement Learning Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Machine Learning: Reinforced Learning Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Machine Learning: Learning Types Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Machine Learning: Learning Types Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  What is Learning? “To gain knowledge or understanding of, or skill in by study, instruction or experience‘’ Learning a set of new facts. Learning HOW to do something. Improving the ability of something already learned. Assoc. Prof. Hanaa Salem Marie Fall 2024/2025  What is Machine Learning? Learning Trained algorithm machine TRAINING DATA Answer Query Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Steps in machine learning 1) Data collection. 2) Representation. 3) Modeling. 4) Estimation. 5) Validation. 6) Apply learned model to new “test” data Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  General structure of a learning system Learning system Data Learning Feed-back Process Problem Solving Teacher Results Performance Evaluation Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Advantages and Disadvantage of ML Advantages of ML Disadvantages of ML 1) Solving vision problems 1) Application specific through statistical algorithms. inference. 2) Real world problems have 2) Intelligence from the too many variables and common sense AI. sensors might be too noisy. 3) Reducing the constraints over time achieving 3) Computational complexity. complete autonomy. Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Types of machine Learning Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Types of Supervised Learning Algorithm ❑ Supervised learning is typically divided into two main categories: ✓ Regression and classification. ✓ In regression, the algorithm learns to predict a continuous output value, such as the price of a house or the temperature of a city. ✓ In classification, the algorithm learns to predict a categorical output variable or class label, such as whether a customer is likely to purchase a product or not. Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Types of Supervised Learning Algorithm ❑ Regression ✓Regression is a supervised learning technique used to predict continuous numerical values based on input features. It aims to establish a functional relationship between independent variables and a dependent variable, such as predicting house prices based on features like size, bedrooms, and location. ✓The goal is to minimize the difference between predicted and actual values using algorithms like Linear Regression, Decision Trees, or Neural Networks, ensuring the model captures underlying patterns in the data. Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Types of Supervised Learning Algorithm ❑ Classification ✓Classification is a type of supervised learning that categorizes input data into predefined labels. It involves training a model on labeled examples to learn patterns between input features and output classes. In classification, the target variable is a categorical value. For example, classifying emails as spam or not. ✓ The model’s goal is to generalize this learning to make accurate predictions on new, unseen data. Algorithms like Decision Trees, Support Vector Machines, and Neural Networks are commonly used for classification tasks. ✓NOTE: There are common Supervised Machine Learning Algorithm that can be used for both regression and classification task. Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Applications of Machine Learning Medical diagnosis Drug discovery Iris verification Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Applications of Machine Learning Hand-written digits Speech Radar Imaging Recognition Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Applications of Machine Learning Face Recognition Signature Fingerprint Verification Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Applications of Machine Learning Traffic Monitoring Target Recognition Robotics vision Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Resources: Datasets UCI Repository: http://www.ics.uci.edu/~mlearn/MLRepository.html UCI KDD Archive: http://kdd.ics.uci.edu/summary.data.applicatio n.html Statlib: http://lib.stat.cmu.edu/ Delve: http://www.cs.utoronto.ca/~delve/ Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Readings https://www.youtube.com/watch?v=rLOyrWV8gmA  Machine Learning Problems Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Machine Learning Framework Apply a prediction function to a feature representation of the image to get the desired output: f( ) = “apple” f( ) = “tomato” f( ) = “cow”  The Machine Learning Framework y = f(x) output prediction Image function feature Training: given a training set of labeled examples {(x1,y1), …, (xN,yN)}, estimate the prediction function f by minimizing the prediction error on the training set Testing: apply f to test example x and output the predicted value y = f(x)  Training Training The machine learning framework Labels Training Images Image Learned Training Features model Testing Image Learned Prediction Features model Test Image  Features ❑Raw pixels ❑Histograms ❑GIST descriptors ❑… Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Generalization Components of generalization error Bias: how much the average model over all training sets differ from the true model? Error due to inaccurate assumptions/simplifications made by the model Variance: how much models estimated from different training sets differ from each other Underfitting: model is too “simple” to represent all the relevant class characteristics High bias and low variance High training error and high test error Overfitting: model is too “complex” and fits irrelevant characteristics (noise) in the data Low bias and high variance Low training error and high test error Assoc. Prof. Hanaa Salem Marie Spring 2023/2024  Readings https://www.youtube.com/watch?v=rLOyrWV8gmA  Thank You Associate Prof. Hanaa Salem Marie 2024/2025