Machine Learning Course - Al-Baha University PDF

Machine Learning Al-Baha University Faculty of Computng Informaton AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.1 Course Overview Chapter 0 AlBaha University...

Machine Learning Al-Baha University Faculty of Computng Informaton AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.1 Course Overview Chapter 0 AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.2 Contents Course Descripton. Course prerequisites. Course Administraton. References. Course Outline. Course Grade Distributon. AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.3 Course Description This course provides an overview of Machine Learning (ML). ML algorithms is a key element to develop intelligent systems and analyze data in science and engineering. ML engines enable intelligent technologies such as Siri, Kinect or Google self-driving car, to work properly. This course provides an introducton to the fundamental methods at the core of modern ML. It covers theoretcal foundatons as well as essental algorithms for supervised and unsupervised learning. AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.4  Course Description: Lab Laboratory exercises will be used to demonstrate problem-solving techniques. Classes on theoretcal and algorithmic aspects are complemented by practcal lab sessions. Lab work demonstrates the power of ML algorithms in practce. Supervised and unsupervised ML AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.5 Course prerequisites The prerequisites for this course: Artfcial Intelligence AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.6 Course Administration (F61) CRN. 21937 Dr. Fahad Ghamdi (lectures) [email protected] Ofce: S612 e Lecture tmetable: Tuesday Female (8:00 - 9:50 AM) Mr. ……(lab) Ofce hours: TBA Lab tmetable: TBA Lecture slides and Assignments will be uploaded on Blackboard. htps://albaha.blackboard.com/webapps/login/ AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.7 References Main Textbook: Chris Bishop; Patern Recogniton and Machine Learning Alternate References: T. Poggio and S. Smale. The Mathematcs of Learning: Dealing with Data. Notces of the AMS, 2003 Pedro Domingos. A few useful things to know about machine learning. Communicatons of the ACM CACM Homepage archive. Volume 55 Issue 10, October 2012 Pages 78-87. T. Haste, R. Tibshirani, and J. Friedman. The Elements of Statstcal Learning: Predicton, Inference and Data Mining. Second Editon, Springer Verlag, 2009 (available for free from the author's website). NOTE: Some material was derived from online sources or authored by other Faculty members. AlBaha University Faculty of Computer Science and Informaton Technology Course Coordinator: Dr. Abdulkareem Aodah 1.8  Course Outline Topics to be Covered List of Topics No. of Weeks Contact hours Introducton to Machine Learning tools and frameworks 1 6 Math Refresher 2 4 Data Pre-processing 1 2 Regression 2 4 Supervised learning- Classifcaton 2 4 Reinforcement learning 1 2 Deep Learning 2 4 Project presentatons 1 2 AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.9  Course Grade Distribution Assessment Method Proporton of Total Assessment Date of providing the material Assignments/Homework 5% Periodically Quiz 5% Periodically Mid Term e0 % Week 7 Lab Exam e0% Week 12 4 Project Presentatons 12 0% Week 12 4 Final Exam 40% According to Final Exam Schedule AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.10 Machine learning: what? Study of making machines learn a concept without having to explicitly program it. Constructng algorithms that can: learn from input data, and be able to make predictons. fnd interestng paterns in data. Analyzing these algorithms to understand the limits of ‘learning’ AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.11 The Concept of Learning in ML System Learning = improving with experience at some task Improve over task T, With respect to performance measure, P Based on Experience, E AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.12 AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.13 Machine learning: why? We are smart programmers, why can’t we just write some code with a set of rules to solve a partcular problem? Write down a set of rules to code to distnguish these two faces: AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.14 Related Fields data mining control theory statstcs decision theory informaton theory machine learning cognitve science databases psychological models evolutonary neuroscience models Machine learning is primarily concerned with the accuracy and efectveness of the computer system. Machine learning: problems in the real world Recommendaton systems (Netlix, Amazon, Overstock) Stock predicton (Goldman Sachs, Morgan Stanley) Risk analysis (Credit card, Insurance) Face and object recogniton (Cameras, Facebook, Microsof) Speech recogniton (Siri, Cortana, Alexa, Dragon) Search engines and content fltering (Google, Yahoo, Bing) AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.16 Machine learning: how? so…. how do we do it? This is what we will focus on in this class! AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.17 This course We will learn: Study a predicton problem in an abstract manner and come up with a soluton which is applicable to many problems simultaneously. Diferent types of paradigms and algorithms that have been successful in predicton tasks. AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.18  Let’s get started! AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.19 Machine Learning: the basics… Data: A set of data records (also called examples, instances or cases) described by k atributes: A1, A2, … Ak. a class: Each example is labelled with a pre-defned class. Goal: To learn a classifcaton model from the data that can be used to predict the classes of new (future, or test) cases/instances. AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.20 Machine Learning: the basics… Data Set Example AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.21 Machine Learning: the basics… Learn a classifcaton model from the data Use the model to classify future loan applicatons into Yes (approved) and No (not approved) What is the class for following case/instance? AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.22 Learning system model Learning (training): Learn a model using the training data Testng: Test the model using unseen test data to assess the model accuracy Input Testng Learning Samples Method System Training AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.23 Training and Testing AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.24 Training and Testing Training is the process of making the system able to learn. No free lunch rule: Training set and testng set come from the same distributon Need to make some assumptons or bias AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.25 Performance There are several factors afectng the performance: Types of training provided The form and extent of any inital background knowledge The type of feedback provided The learning algorithms used Two important factors: Modeling Optmizaton AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.26 Algorithms Supervised learning: classifcaton is seen as supervised learning from examples. Supervision: The data (observatons, measurements, etc.) are labeled with pre-defned classes. It is like that a “teacher” gives the classes (supervision). Test data are classifed into these classes too. Unsupervised learning (clustering) Class labels of the data are unknown Given a set of data, the task is to establish the existence of classes or clusters in the data Semi-supervised learning develop mathematcal models from incomplete training data, where a porton of the sample input doesn't have labels. Reinforcement learning Decision making (robot, chess machine) AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.27 Algorithms AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.28 Machine Learning Structure Supervised learning AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.29 Machine Learning Structure Unsupervised learning AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.30 What Are We Seeking? Supervised: Low E-out or maximize probabilistc terms E-in: for training set E-out: for testng set Unsupervised: Minimum quantzaton error, Minimum distance, MAP, MLE(maximum likelihood estmaton) AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.31 What Are We Seeking? AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.32 Learning Techniques Supervised learning categories and techniques Linear classifer (numerical functons) Parametric (Probabilistc functons) Naïve Bayes, Gaussian discriminant analysis (GDA), Hidden Markov models (HMM), Probabilistc graphical models Non-parametric (Instance-based functons) K-nearest neighbors, Kernel regression, Kernel density estmaton, Local regression Non-metric (Symbolic functons) Classifcaton and regression tree (CART), decision tree Aggregaton Bagging (bootstrap + aggregaton), Adaboost, Random forest AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.33  Learning Techniques Linear classifer , where w is an d-dim vector (learned) Techniques: Perceptron Logistc regression Support vector machine (SVM) Ada-line Mult-layer perceptron (MLP) AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.34 Learning Techniques Training Testng Error rate: 0.10 Error rate: 0.156 AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.35 Learning Techniques Using logistc regression Training Testng Error rate: 0.11 Error rate: 0.145 AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.36 Learning Techniques Non-linear case Support vector machine (SVM): Linear to nonlinear: Feature transform and kernel functon AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.37 Learning Techniques Unsupervised learning categories and techniques Clustering K-means clustering Spectral clustering Density Estmaton Gaussian mixture model (GMM) Graphical models Dimensionality reducton Principal component analysis (PCA) Factor analysis AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.38 Machine Learning: Examples A closer look at some predicton problems… Handwriten character recogniton: MINST Dataset Labels Spam fltering: Emails Dataset Labels Object recogniton: 1.39 AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning Spam Filtering Example Commonalites in a predicton problem: Example: Spam Filtering Spam- is all mail the user does not want to receive and has not asked to receive T: identfy Spam Emails P: % of spam emails that were fltered % of ham/non-spam emails that were incorrectly fltered-out. E: a database of emails that were labelled by users AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.40 The Learning Process AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.41 The Learning Process in our Example AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.42 Data Set AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.43 Step 4: Model Learning Database Learner Model Data Set Inducer Inducton Algorithm AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.44 Step 5: Model Testing Database Learner Model Data Set Inducer Inducton Algorithm AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.45 Leaning Algorithm AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.46 AlBaha University Faculty of Computer Science and Informaton Technology Dr. Sonia LAJMI 1.47 AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.48 AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.49 AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.50 AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.51 Linear Classifers AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.52 Linear Classifers AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.53 Linear Classifers When a new email is sent: 1. We frst place the new email in the space 2. Classify it according to the subspace in which it resides AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.54 Linear Classifers AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.55 Linear Classifers AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.56 Linear Classifers AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.57 Linear Classifers AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.58 Maximum Margin AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.59 Maximum Margin AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.60 Linear Classifers AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.61 Linear Classifers AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.62 Ideally, the best decision boundary should be the one which an optmal performance such as the following… AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.63 Linear Classifers No Linea classifer can cover all instances AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.64  However, our satsfacton is premature because the central aim of designing a classifer is to correctly classify novel input issue of generalizaton AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.65  Which one? AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.66 Evaluating what has been Learned 1. We randomly select porton of the data to be used for training 2. Train the model on the training set 3. Once the model is trained, we run the model on the remaining instances (test set) to see how it performs AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.67 The non-Linearly separable case AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.68 The non-Linearly separable case AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.69 The non-Linearly separable case AlBaha University Faculty of Computer Science and Informaton Technology Course: Machine Learning 1.70  Thank You AlBaha University Faculty of Computer Science and Informaton Technology Course: Data Structures and Algorithms 1.71