Introduction to Machine Learning 1 PDF

Introduction to Machine Learning Introduction to Data Science (G6085) Dr Benjamin Evans 1 Mid-Module Evaluation We are now roughly halfway through the module which presents a good opportunity to hear how you think it i...

Introduction to Machine Learning Introduction to Data Science (G6085) Dr Benjamin Evans 1 Mid-Module Evaluation We are now roughly halfway through the module which presents a good opportunity to hear how you think it is going so that we can make any adjustments necessary for the second half and future years. Please take a few minutes to let us know how its going by filling out the brief Mid-Module Evaluation survey, which you can find under "Quizzes" on the Canvas module, or directly linked here. The survey is anonymous and will help us improve the course for you. The survey is now open and closes next Sunday 10/11/24 at 4pm. I look forward to reading your views and thank you in advance for taking the time to provide them. 2 Mid-Module Evaluation https://canvas.sussex.ac.uk/courses/33205/quizzes/49785 The data science process Week Topic Data Real 1 Data, the data science world Solution / process & data science tools Product 2 Statistics & Probability Theory Communication 3 Exploratory Data Analysis / Data and Visualisation Collection Visualisation 4 Statistical Inference / Hypothesis Testing Data 5 Data collection, wrangling & Wrangling cleaning / Pre-processing 6 ~~ No lectures ~~ Exploratory Mathematical Data 7-11 Machine Learning: Data Analysis Modelling Supervised Learning, Cleaning Unsupervised Learning, Model evaluation & selection Statistics & Probability Theory 4 What is Machine Learning? A subset of AI involving computer programmes, often referred to as models, which: are trained to achieve some task, without requiring programming explicit instructions for how to do it rather they enable computers to perform pattern recognition. these patterns are learned from some example data. A key factor in ML is getting the computer to learn a generalisable mechanism for achieving the task. 5 This Photo by Unknown Author is licensed under CC BY Why is Machine Learning Interesting? Potential to delegate trivial, repetitive or 63 million articles dangerous jobs to machines. Faster processing of information Vast volumes of data are being generated 3 billion users and stored digitally, more than any human could ever hope to look at/learn from. 500 million tweets a day ► Potential for deeper insights to be learned from more data (6,000 per second!) 50 billion webpages Updated November 2024 6 Why Should Machine Learning be Interesting to You? Curiosity – machine learning is tasked with designing frameworks to learn and reason about data without needing to articulate explicit decision rules. It is becoming ubiquitous: you interact with increasingly many ML systems in your day- to-day lives. Combination of programming, applied mathematics and machine learning should fit well within your skillset at graduation. Job prospects are very good in this field at the moment! 7 How do you interact with ML in your life? Let’s find out what you know about ML and how it might be involved in your day-to-day life: PollEv.com/bdevans This will help us to think about different possible tasks, and their requirements. 8 9 Outline for today The rest of the module: content overview The What and Why of machine learning Ingredients of machine learning: tasks, features, and models 10 Learning Outcomes Understand the variety of machine learning tasks such as classification, regression, clustering and dimensionality reduction. Learn what models are. Recognise the importance of features (dimensions) as an inputs to machine learning. 11 About This S e c t i o n o f t h e Module This is an introduction to the field of machine DATA learning. INSIGHTS It aims to provide a unified view of the field and to present key ideas and techniques. We will take a fairly applied, example-driven approach, but will also try to cover important theoretical concepts. 12 Learning Outcomes By the end of this module a successful student should be able to: Demonstrate basic knowledge of several supervised and unsupervised machine learning models including: linear regression, the perceptron, random forest, PCA and K-means clustering. Map machine learning models to tasks based on reasoned arguments. Explain and exploit practical concepts such as cross-validation and learning curves. Use machine learning toolboxes to solve classification/regression problems with real-world data, including pre-processing of the data and incorporating prior knowledge. 13  Use the resources provided! Canvas webpage: ►All teaching materials (lecture notes, lab exercises & solutions, assessment details, additional reading, etc.) will be made available at https://canvas.sussex.ac.uk/courses/33205. ►Module Discussion Forum on Canvas: If you have a question, it is likely someone else is wondering the same thing, so please use this forum to ask any questions and share resources for everyone’s benefit. Labs: ► Are very important - this is where the learning is consolidated! ► Take longer than the 1-hour slot. ► Prepare in advance. ► Attempt the extensions if you can. ► Prepare in advance to make the most out of them Source Materials Some of the good options: C. M. Bishop, Pattern Recognition and Machine Learning, Springer, 2007. https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop- Pattern-Recognition-and-Machine-Learning-2006.pdf S. J. D. Prince. Computer Vision: Models, Learning, and Inference. 2012. http://www.computervisionmodels.com/. S. J. D. Prince. Understanding Deep Learning. 2023. https://udlbook.github.io/udlbook/. K. Murphy. Probabilistic Machine Learning: An introduction. 2021. https://probml.github.io/pml-book/ MP Deisenroth, AA Faisal, CS Ong, Mathematics for Machine Learning 2020. Online version available at: https://mml-book.github.io/book/mml-book.pdf Useful video materials: https://www.coursera.org/course/ml and http://videole ctures.net → search for “machine learning summer school”. 18 Mathematics and Machine Learning Machine learning is underpinned by a variety of mathematical theories and formalisms. Difficult to present without at least some of the relevant mathematics: ► probability theory ► linear algebra ► multivariable calculus ► information theory ► logic and set theory It is important to try to understand the intuition behind the mathematics. This influences the choice of what method to use in which situation. 19 The remainder of the module Week Tuesday Thursday Lecturer 7 Intro to ML SL: Classification BE 8 SL: Linear Regression SL: Extending Linear DR Regression 9 Evaluating performance Model Selection DR 10 UL: Clustering UL: Dimensionality Reduction BE 11 The Perceptron Random Forests BE 20 25  S2T1 S2T2 S2T3 S2T4 S2T5 S2T6 S2T7 S2T8 wij = kri  rj ri  = (1 −  )ri + ri  −1 Object Position 3 1 1 Trained Trained 0.9 Untrained 0.9 Untrained 0.8 0.8 0.7 0.7 Information (bits) Information (bits) 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 0 50 100 150 200 250 0 1 2 3 4 5 6 7 8 9 10 Cell rank Ensemble size Single cell information Multiple cell information 𝑂ሶ = 𝐺𝑎 𝜙 𝐶 − 𝐺𝑑 𝑂 𝐷ሶ = 𝐺𝑑 𝑂 − 𝐺𝑟 𝐷 𝐶ሶ = 𝐺𝑟 𝐷 − 𝐺𝑎 𝑂 26 Architecture 27 Why Machine Learning? Data can be used to perform all kinds of tasks: 1 Identifying whether an email contains irrelevant information (spam) or not (not spam). 28 Why Machine Learning? Data can be used to perform all kinds of tasks: 2 Predicting from satellite images and environmental sensors whether tomorrow will be sunny or partly cloudy or cloudy or rainy or thunderstorm. 29 Why Machine Learning? Data can be used to perform all kinds of tasks: 3 Predicting from satellite images and environmental sensors for tomorrow’s temperature, wind, and humidity values. 10 oC Wi nd : 6mph Humidity : 40% 30 Why Machine Learning? Machine Lea rning: A Probabilistic Qu anti ty: 1 Perspective (Ada ptive Computa tion and Machine Lea rning Serie s) [Hardcover] Yes, I want FRE E One-Day Deliv ery wi th a free tri al of K evi n M urp hy (Au thor) Amazon Prime (3 customer reviews) Data can be used to perform all kinds of tasks: R RP : £48.95 Pr ice: £41.49 & this item Delivered FREE in or S ig n i n to tur n on 1-C li ck ordering. the UK with Super Saver Delivery. See details and conditions 4 Recommending products and services on You S ave: £7.46 (15%) In stock. Di sp atched from an d sol d by Am az on. Gi ft-wr ap available. Trade in Yours For a £23.15 Gi ft Card the basis of customer purchase history and Wa nt it Sat urday , 21 De c.? Or der i t wi thi n 21 hr s 9 m ins an d choo se O ne -D ay D eli ve r y at checko ut. Detai ls O rder ing fo r Christ ma s? To en su re de li very by December Lear n more 24, choo se Supe r Sav er De live ry at checko ut. Read more ab out ho li day delivery. Mo re Buying Cho ices the decisions made by similar customers. 31 new from £3 7.00 1 co llect ible from £48.95 6 use d from £42.03 38 use d & new from £37.00 Have on e to sel l? Share Formats Am azon P rice New from Used from C li ck to open expanded view Share your own customer images K indle Edition £31.12 -- -- Search inside thi s book Hardcover £41. 49 £37. 00 £42.03 S tart re adi ng M achin e Lear ni ng on you r iPad, PC or M ac i nstantl y. Don 't ha ve a free Ki ndle ap p? Get you rs here Specia l Offer To day on ly, wh en you bu y a ph ysical bo ok, you wi ll re ce ive a £1 cred it for K in dle B ooks. Explore tho usand s of bo oks from 99 p i n the K in dle S tore an d start re adi ng i n second s wi th on e of ou r free K in dle re adi ng ap ps for you r smar tphon e, tab let, or PC. Lear n mor e ab out thi s offer. What Other Items Do Customers Buy After Viewing This Item? Bayesian Reasoning and Machine Learning by Davi d B arb er Hardcover (2) £38.25 Pattern Recognition and Machine Learning (Information Science and Statistics) by C hri stoph er M. B isho p Hardcover (10) £63.99 MACHINE LEARNING (Mcgraw-Hill International Edit) by Th om M. M itchel l Paperback (5) £43.34 31 Why Machine Learning? Data can be used to perform all kinds of tasks: 5 Grouping together images according to semantic similarities (clustering or compressing data). 32 Why Machine Learning? Data can be used to perform all kinds of tasks: 6 Visualising high dimensional data in a 2- dimensional space. Saul & Roweis, 2003. 33 34 35 Why Machine Learning? Some problems are difficult or even impossible to formalise as a computer science problem, but humans can provide examples or feedback. Given an image of a handwritten character, which character is this? Character Recognition It is difficult to program a solution to this. 36 Why Machine Learning? Some problems are difficult or even impossible to formalise as a computer science problem, but humans can provide examples or feedback. Given an image of a handwritten character, which character is this? Character Recognition It is difficult to program a solution to this. 37 Why Machine Learning? Some problems are difficult or even impossible to formalise as a computer science problem, but humans can provide examples or feedback. Given an image of an animal, which animal is this? Object Category Recognition It is impossible to program a solution to this. 38 Why Machine Learning? The good news: we have examples. Training by examples Test Machine Dog Learning 39 Tasks, Models, and Features Three main ingredients of machine learning: Tasks: ► problems that require a mapping from data to desired outputs (insights). Features: ► characteristics of the data used to describe domain objects. Models: ► encode the required task mapping. Data (Features) Machine Insights Learning Dog Models 40 41 (Some) Machine Learning Tasks if You have Health Data Classification Binary (C = 2) classification involves separating data into two distinct groups (+ and −) e.g. distinguishing people at risk from heart disease (+), from those not at risk (−) Generalise to C > 2 different classes Regression Involves mapping from data items to real values e.g. quantifying the risk of heart disease on the basis of personal health records Clustering Separating data into different clusters/concepts on the basis of their characteristics e.g. grouping people according to their genetic characteristics Collaborative filtering Identifying rules or associations from data e.g. “recommending” treatments based on patient records and diseases of similar patients Dimensionality reduction Visualising the data or preprocessing for better model training Reinforcement Learning Learning a policy to map the environment to behaviours in order to maximise a reward 42 Teaching your models : Supervised Learning There are different teaching paradigms in machine learning. In supervised machine learning, each element of the training data has associated labels. ► These labels are the appropriate “output” for this data example e.g. binary classification: patient data already labelled with “at risk of heart disease” or “not at risk of heart disease”. ► Model should generalise from training data i.e. be good at predicting output for unseen data. Supervised Learning 43 Teaching your models : Unsupervised Learning Unsupervised machine learning does not require labels. Learns structure and patterns based on the similarities and differences in the data features of the training examples. e.g. clustering: grouping people on basis of genetic similarity in order to discover interesting sub-groups or find a compressed representation. Unsupervised Learning 44 Teaching your models : Degrees of supervision The choice is not always full/no supervision. Semi-supervised machine learning methods use labels on a subset of the data. This is particularly relevant when the labels are expensive. 45 Data and Features Features are the inputs to machine learning: ► Used to describe data objects ► Represent particular characteristics of an instance ► May be numerical (e.g. age, weight, income), boolean (e.g. is employed, is male), ordinal (e.g. SES, satisfaction), or nominal (e.g. nationality). A model is only as good as the information it sees: ► Choice of features is therefore extremely important ► Techniques for removing redundant features or transforming features in various ways are often crucial 46 Machine Learning Models Models form the central concept in machine learning: ► A framework for making predictions from features ► With parameters learned from data ► Models encode the mapping needed to solve a task We will talk about several different models during this module. 47 Quiz Time! Go to https://pollev.com/bdevans 10 oC Wind : 6mph Humidity : 40% Which machine learning task is this? A Classification B Regression C Clustering D Collaborative Filtering E Dimensionality Reduction 48 49 50 51 Quiz Time! Go to https://pollev.com/bdevans Which machine learning task is this? A Classification B Regression C Clustering D Collaborative Filtering E Dimensionality Reduction 52 53 54 55 Summary Introduced Machine Learning Considered different types of ML task, including: Classification Regression Clustering Dimensionality Reduction Explored what makes some problems hard for traditional approaches Discussed the main paradigms of Machine Learning: Supervised Learning Unsupervised Learning Semi-supervised Learning Next lecture: A closer look at classifiers and how to evaluate them! 56