Introduction to Machine Learning PDF

**University of Science and Technology** **Faculty of Computer Science and Information Technology** **Department of Computer Science.... Semester 8** **Subject: Introduction to Machine Learning** Lecture (1): Basic Definitions and Concepts **Instructor: Prof. Noureldien Abdelrahman Noureldien D...

**University of Science and Technology** **Faculty of Computer Science and Information Technology** **Department of Computer Science.... Semester 8** **Subject: Introduction to Machine Learning** Lecture (1): Basic Definitions and Concepts **Instructor: Prof. Noureldien Abdelrahman Noureldien Date: 1-3-2023** **1.1 What Is Machine Learning?** *Machine Learning is the science (and art) of programming computers so they can learn from data.* Here is a slightly more general definition: ***Machine Learning is the field of study that gives computers the ability to learn without being explicitly programmed.*** And a more engineering-oriented one: ***A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.*** For example, your spam filter is a Machine Learning program that can learn to flag spam given examples of spam emails (e.g., flagged by users) and examples of regular (non-spam, also called "ham") emails. The examples that the system uses to learn are called the training set. Each training example is called a training instance (or sample). In this case, the task T is to flag spam for new emails, the experience E is the training data, and the performance measure P needs to be defined; for example, you can use the ratio of correctly classified emails. This particular performance measure is called accuracy and it is often used in classification tasks. **1.2 Why Use Machine Learning?** **1.2.1 Traditional Programming Technique** Consider writing a spam filter using traditional programming techniques (Figure 1-1): 1\. First you would look at what spam typically looks like. You might notice that some words or phrases (such as "4U," "credit card," "free," and "amazing") tend to come up a lot in the subject, and so on. 2\. You would write a detection algorithm for each of the patterns that you noticed, and your program would flag emails as spam if a number of these patterns are detected. 3\. You would test your program, and repeat steps 1 and 2 until it is good enough. Since the problem is not trivial, your program will likely become a long list of complex rules---pretty hard to maintain. **1.2.2 Machine Learning Technique** In contrast, a spam filter based on Machine Learning techniques automatically learns which words and phrases are good predictors of spam by detecting unusually frequent patterns of words in the spam examples compared to the ham examples (Figure 1-2). The program is much shorter, easier to maintain, and most likely more accurate.  **1.3 Machine Learning is a Multi-Disciplinary Field** Machine Learning has concepts that have been derived and borrowed from multiple fields over a period of time since its inception, making it a true multi-disciplinary or inter-disciplinary field. **[Figure 1-4 should give you a good idea with regard to the major fields that overlap with Machine Learning based on concepts, methodologies, ideas, and techniques]**. **1.4 Types of Machine Learning Systems** There are so many different types of Machine Learning systems that it is useful to classify them in broad categories based on: ***- Supervised or unsupervised*** Whether or not they are trained with human supervision (supervised, unsupervised, semi supervised, and Reinforcement Learning) ***- Batch or Online*** **[Whether or not they can learn]** incrementally on the fly (online versus batch learning) ***- Instance-Based Versus Model-Based Learning*** Whether they ***[work by]* [simply comparing new data points to known data points]**, or instead ***[work by]*** **[detect patterns in the training data and build a predictive model]**. **1.4.1 Supervised/Unsupervised Learning** Machine Learning systems can be classified according to the amount and type of supervision **[they get during training]**. There are four major categories: supervised learning, unsupervised learning, semi-supervised learning, and Reinforcement Learning. 1. **Supervised learning** In supervised learning, the training data you feed to the algorithm includes **the desired solutions, called labels (**Figure 1-5).  a. ***[Supervised Tasks ]*** 1. [ ***Classification***]. **[This is a typical supervised learning task]**. The spam filter is a good example of this: it is trained with many example emails along with their class (spam or ham), and it must learn how to classify new emails. 2. ***[Regression]***: **Another typical task is to predict a target numeric value**, such as the price of a car, given a set of *[features]* (mileage, age, brand, etc.) **[called predictors]**. This sort of task is called regression (Figure 1-6). To train the system, you need to give it many examples of cars, including both their predictors and their labels (i.e., their prices). In Machine Learning **[an attribute is a data]** type (e.g., "Mileage"), while a feature has several meanings depending on the context, **[but generally means an attribute plus its value]** (e.g., "Mileage = 15,000"). Many people use the words attribute and feature interchangeably. Here are some of the most important supervised learning algorithms k-Nearest Neighbors Linear Regression Logistic Regression Support Vector Machines (SVMs) Decision Trees and Random Forests Neural networks