ML.ch1.pdf

Machine Learning Prepared By: Dr. Sara Sweidan The origins of machine learning The origins of machine learning The origins of machine learning  The origins of machine learning Computing Available power data St...

Machine Learning Prepared By: Dr. Sara Sweidan The origins of machine learning The origins of machine learning The origins of machine learning  The origins of machine learning Computing Available power data Statistical methods  The origins of machine learning Machine learning is known as the development of computer algorithms to transform data into intelligent action The origins of machine learning Virtually all data mining involves the use of machine learning, but not all machine learning involves data mining. For example, you might apply machine learning to data mine automobile traffic data for patterns related to accident rates; on the other hand, if the computer is learning how to drive the car itself, this is purely machine learning without data mining.  The origins of machine learning Machine Learning definition Arthur Samuel (1959). Machine Learning: Field of study that gives computers the ability to learn without being explicitly programmed. Tom Mitchell (1998) Well-posed Learning Problem: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E  Machine-learning definition Machine learning is concerned with the development of algorithms for building mathematical models that allow a computer to learn from historical data and experiences to make decisions. the historical data are known as training data. Machine learning successes A survey of recent success stories includes several prominent applications: Identification of unwanted spam messages in e-mail Segmentation of customer behavior for targeted advertising Forecasts of weather behavior and long-term climate changes Reduction of fraudulent credit card transactions Actuarial estimates of financial damage of storms and natural disasters Prediction of popular election outcomes Development of algorithms for auto-piloting drones and self-driving cars Optimization of energy use in homes and office buildings Projection of areas where criminal activity is most likely Discovery of genetic sequences linked to diseases Machine learning limitations It has very little flexibility to extrapolate outside of the strict parameters it learned and knows no common sense. Should be extremely careful to recognize exactly what the algorithm has learned before setting it loose in real-world settings. Without a lifetime of past experiences to build upon, computers are also limited in their ability to make simple common sense inferences about logical next steps. Machine Learning - Grew out of work in AI - New capability for computers Examples: - Database mining Large datasets from growth of automation/web. E.g., Web click data, medical records, biology, engineering - Applications can’t program by hand. E.g., Autonomous helicopter, handwriting recognition, most of Natural Language Processing (NLP), Computer Vision. - Self-customizing programs E.g., Amazon, Netflix product recommendations - Understanding human learning (brain, real AI).  How machines learn Data storage utilizes observation, memory, and recall to provide a factual basis for further reasoning. Abstraction involves the translation of stored data into broader representations and concepts. Generalization uses abstracted data to create knowledge and inferences that drive action in new contexts. Evaluation provides a feedback mechanism to measure the utility of learned knowledge and inform potential improvements.  Data storage All learning must begin with data. Humans and computers alike utilize data storage as a foundation for more advanced reasoning. In a human being, this consists of a brain that uses electrochemical signals in a network of biological cells to store and process observations for short- and long-term future recall. Computers have similar capabilities of short- and long-term recall using hard disk drives, flash memory, and random access memory (RAM) in combination with a central processing unit (CPU).  Abstraction This work of assigning meaning to stored data occurs during the abstraction process, in which raw data comes to have a more abstract meaning. This type of connection, say between an object and its representation During a machine's process of knowledge representation, the computer summarizes stored raw data using a model, an explicit description of the patterns within the data. There are many different types of models. You may be already familiar with some. Examples include: Mathematical equations Relational diagrams such as trees and graphs Logical if/else rules Groupings of data known as clusters  Abstraction The process of fitting a model to a dataset is known as training. When the model has been trained, the data is transformed into an abstract form that summarizes the original information. Note, You might wonder why this step is called training rather than learning. First, note that the process of learning does not end with data abstraction; the learner must still generalize and evaluate its training. Second, the word training better connotes the fact that the human teacher trains the machine student to understand the data in a specific way.  Generalization The term generalization describes the process of turning abstracted knowledge into a form that can be utilized for future action, on tasks that are similar, but not identical, to those it has seen before. In generalization, the learner is tasked with limiting the patterns it discovers to only those that will be most relevant to its future tasks. the algorithm will employ heuristics, which are educated guesses about where to find the most useful inferences.  Generalization The algorithm is said to have a bias if the conclusions are systematically erroneous, or wrong in a predictable manner.  Evaluation Bias is a necessary evil associated with the abstraction and generalization processes inherent in any learning task. In order to drive action in the face of limitless possibility, each learner must be biased in a particular way. Consequently, each learner has its weaknesses and there is no single learning algorithm to rule them all. Therefore, the final step in the generalization process is to evaluate or measure the learner's success in spite of its biases and use this information to inform additional training if needed. Generally, evaluation occurs after a model has been trained on an initial training dataset. Then, the model is evaluated on a new test dataset in order to judge how well its characterization of the training data generalizes to new, unseen data. It’s worth noting that it is exceedingly rare for a model to perfectly generalize to every unforeseen case.  Evaluation In parts, models fail to perfectly generalize due to the problem of noise, a term that describes unexplained or unexplainable variations in data. Noisy data is caused by seemingly random events, such as: Measurement error due to imprecise sensors that sometimes add or subtract a bit from the readings. Issues with human subjects, such as survey respondents reporting random answers to survey questions, in order to finish more quickly. Data quality problems, including missing, null, truncated, incorrectly coded, or corrupted values. Phenomena that are so complex or so little understood that they impact the data in ways that appear to be unsystematic. Trying to model noise is the basis of a problem called overfitting.  Machine learning in practice To apply the learning process to real-world tasks, we'll use a five-step process. any machine learning algorithm can be deployed by following these steps: 1- data collection 2- data preparation 3- model training 4- model evaluation 5- model improvement Machine learning in practice  Machine learning in practice Matching Types of Types of input data input data algorithms to algorithms  Machine learning in practice Types of input data: Unit of observation ➔ dataset matrix Dataset : examples , features E.g. Unit of observation: patient Features: biomarkers Examples: cases Machine learning in practice  Machine learning in practice Types of input data: Features also come in various forms. If a feature represents a characteristic measured in numbers, it is unsurprisingly called numeric. Alternatively, if a feature is an attribute that consists of a set of categories, the feature is called categorical or nominal. A special case of categorical variables is called ordinal, which designates a nominal variable with categories falling in an ordered list. Some examples of ordinal variables include clothing sizes such as small, medium, and large; or a measurement of customer satisfaction on a scale from "not at all happy" to "very happy." It is important to consider what the features represent, as the type and number of features in your dataset will assist in determining an appropriate machine learning algorithm for your task.  Machine learning in practice Types of algorithms : Descriptive Model Predictive Model  Machine learning in practice Predictive model: (supervised learning) Regression Classification (numeric prediction) (categorical prediction)  Machine learning in practice Predictive model: (supervised learning) A predictive model is used for tasks that involve, as the name implies, the prediction of one value using other values in the dataset. The learning algorithm attempts to discover and model the relationship between the target feature (the feature being predicted) and the other features. -Supervised ML for predicting which category is known as classification. -predicated feature is a categorical feature known as class. -class have two or more levels may be ordinal. e.g. An e-mail message is spam, a person has cancer, a football team will win or lose, an applicant will default on a loan  Breast cancer (malignant, benign) Classification 1(Y) Discrete valued Malignant? output (0 or 1) 0(N) Tumor Size Tumor Size  Machine learning in practice Predictive model: (supervised learning) To predict such numeric values, a common form of numeric prediction fits linear regression models to the input data. Although regression models are not the only type of numeric models, they are, by far, the most widely used. Regression methods are widely used for forecasting, as they quantify in exact terms the association between inputs and the target, including both, the magnitude and uncertainty of the relationship.  Machine learning in practice descriptive model: (unsupervised learning) Clustering Pattern discovery  Machine learning in practice Descriptive model: (unsupervised learning) -summarizing data in a new and interesting ways. in a descriptive model, no single feature is more important than any other. In fact, because there is no target to learn, the process of training a descriptive model is called unsupervised learning - Pattern discovery: is used to identify useful associations within data. (market basket anaylsis). the goal is to identify items that are frequently purchased together, -clustering (segmentation analysis): that identifies groups of individuals with similar behavior or demographic information.  Supervised Learning x2 x1  Unsupervised Learning x2 x1  Machine learning in practice Matching input data to algorithms: Model Learning task Supervised learning algorithms Nearest neighbor Naïve bayes Classification Decision tree Classification rule learner Linear regression Regression tree Numeric prediction Model trees Neural network Dual use Support vector machine  Machine learning in practice Matching input data to algorithms: Model Learning task Unsupervised learning algorithms Association rules Pattern detection K-means clustering Clustering  “A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.” Suppose your email program watches which emails you do or do not mark as spam, and based on that learns how to better filter spam. What is the task T in this setting? T Classifying emails as spam or not spam. E Watching you label emails as spam or not spam. P The number (or fraction) of emails correctly classified as spam/not spam. None of the above—this is not a machine learning problem. You’re running a company, and you want to develop learning algorithms to address each of two problems. Problem 1: You have a large inventory of identical items. You want to predict how many of these items will sell over the next 3 months. Problem 2: You’d like software to examine individual customer accounts, and for each account decide if it has been hacked/compromised. Should you treat these as classification or as regression problems? Treat both as classification problems. Treat problem 1 as a classification problem, problem 2 as a regression problem. Treat problem 1 as a regression problem, problem 2 as a classification problem. Treat both as regression problems. Of the following examples, which would you address using an unsupervised learning algorithm? (Check all that apply.) Given email labeled as spam/not spam, learn a spam filter. Given a set of news articles found on the web, group them into set of articles about the same story. Given a database of customer data, automatically discover market segments and group customers into different market segments. Given a dataset of patients diagnosed as either having diabetes or not, learn to classify new patients as having diabetes or not.