Intro to Machine Learning

• Machine learning empowers computers to learn from data without explicit programming.

Machine Learning Definitions

• Machine learning involves programming computers to optimize performance using data or experience.
• Learning enhances computer programs automatically through experience.
• A model can be a mathematical expression, equation, graph, rule set, or any structure used for prediction, description, or knowledge extraction from data.

The Essence of Learning in Machine Learning

• A program learns if its performance (P) on a task (T) improves with experience (E).
• Example: Handwriting recognition involves recognizing words in images (T), measured by correct classification rate (P), using a dataset of handwritten words (E).
• Example: Robot driving learns to navigate highways (T), measured by distance traveled before error (P), using image and steering data from a human driver (E).
• Example: Chess playing improves win rate (P) against opponents (T) by practicing against itself (E).
• A machine learning program, also known as a learner, improves from experience.

Machine Learning Process

• Data storage, abstraction, generalization, and evaluation are the four fundamental components.

Components of the Learning Process

• Data storage is a core component, enabling advanced reasoning through storing and retrieving large datasets.
• Abstraction extracts knowledge from stored data by creating general concepts and models, including training models on datasets.
• Generalization turns knowledge into a form applicable to similar future tasks by identifying relevant data properties.
• Evaluation provides feedback to measure the learned knowledge's utility and drive improvements.

Machine Learning Applications

• Applying machine learning to large databases is known as data mining, constructing simple, valuable models.
• Retail: Studying consumer behavior.
• Finance: Building models for credit applications, fraud detection, and stock market analysis.
• Manufacturing: Optimization, control, and troubleshooting.
• Medicine: Medical diagnosis.
• Telecommunications: Network optimization and service quality maximization through call pattern analysis.
• Science: Analyzing large datasets in physics, astronomy, and biology.
• Artificial intelligence: Teaching systems to adapt without pre-programmed solutions.
• Vision, speech recognition, and robotics: Finding solutions to complex problems.
• Computer-controlled vehicles: Steering correctly on various roads.
• Games: Developing programs for chess, backgammon, and Go.

Data Types and Forms

• Unit of observation: The smallest entity with measured properties of interest.
• Examples: person, object, time point, geographic region, or measurement.

Examples and Features

• "Example": A recorded instance of the unit of observation, also known as an "instance," "case," or "record."
• "Feature": A recorded property or characteristic of examples, also known as "attribute" or "variable."
• Cancer detection includes patients as units, cancer patients as examples, and gender, age, blood pressure, and pathology reports as features.
• Pet selection has persons as units, pet owners as examples, and age, home region, and family income as features.
• Spam email identification uses email messages as units, specific messages as examples, and words used in the messages as features.
• Examples and features are commonly organized in a matrix format.

Data Forms

• Numeric data: Features measured in numbers.
• Categorical data: Attributes with a limited number of values based on qualitative properties. Also referred to as "nominal" data.
• Ordinal data: Nominal variables with categories in a specific order.
• Example: "year," "price," and "mileage" are numeric, while "model," "color," and "transmission" are categorical.

Machine Learning Problem Classes

• Learning associations, classification, and regression are fundamental classes.

Learning Associations

• Association rule learning discovers interesting relationships between variables in large databases, called "association rules."
• Supermarket chain analysis identifies patterns in customer purchases, such as customers buying onions and potatoes also buying hamburger.
• Association rules takes the form X ⇒ Y, for "if people buy X then they also buy Y."
• Rules like these are used for cross-selling, promotional pricing, and product placement
• The conditional probability of the form P(Y ∣X) estimates the likelihood of a customer buying product Y, given they have already bought product X.
• Factors like customer attributes (gender, age, and marital status) can be expressed by P(Y ∣X, D).
• Algorithms for generating association rules: Apriori, Eclat, and FP-Growth.

Classification

• In machine learning, classification identifies the category a new observation belongs to, using a training set of data with known category memberships.
• A problem can be posed as follows: If we have some new data, say “Score1 = 25” and “Score2 = 36”, what value should be assigned to “Result” corresponding to the new data; in other words, to which of the two categories or classes the new observation should be assigned?
• Optical character recognition, face recognition, and speech recognition are real-life examples.
• In medical diagnosis, inputs are patient information, and classes are illnesses.
• Classification rules aid knowledge extraction, compression, and various decision-making processes.

Classification Rules

• Rules can be used to classify patients as low-risk or high-risk based on variables like blood pressure and age.
• Credit card companies classify applicants based on annual salary and age.
• Astronomers label distant objects as stars, galaxies, or nebulas using digital images.
• A discriminant is a rule or function that assigns labels to new observations.
• Discriminant Example:
• IF Score1 + Score2 ≥ 60, THEN “Pass” ELSE “Fail”.
• IF Score1 ≥ 20 AND Score2 ≥ 40 THEN “Pass” ELSE “Fail”.
• Examples:
• Logistic regression
• Naive Bayes algorithm
• k-NN algorithm
• Decision tree algorithm
• Support vector machine algorithm
• Random forest algorithm

Classes

• Classification problems classify examples into categories.
• Classifications involve real-valued or discrete input variables.
• Two-class problems are called "binary classification," and those with more than two classes are "multi-class."
• Assigning multiple classes to an example is known as "multi-label classification."

Regression

• Predictions: Predicting numeric variable values based on observations.
• Prediction values: integers or floating-point numbers.
• Input: Discrete or real-valued independent values.

General Approach

• A model presents a mathematical relation where y = f(x, θ).
• "f(x, θ)" = regression function.
• Machine learning algorithm optimizes the parameters in the set θ, minimizing the error.
• Example:

Price = a0 + a1 × (Age) + a2 × (Distance) + a3 × (Weight)

Regression Models

• Simple Linear Regression: one continuous independent variable
• Multivariate Linear Regression: more than one independent variable
• Polynomial Regression: one continuous independent variable x
• Logistic Regression: dependent variable is binary

Types of Learning

• In general, machine learning algorithms are broken into three types of learning: Supervised, unsupervised, and reinforcement

Supervised Learning

• Task: Learning function to map input to output using input-output pairs.
• Training: Each example is a pair of input and desired output.
• Process: Trains data, produces a function to map new examples, and determines correct class labels for new instances.
• Both classification and regression problems are supervised learning problems.
• Supervised learning can be thought of as a teacher guiding the learning process with correct answers.
• An algorithm iteratively predicts the training data, is corrected and then stops after achieving the required performance.

Unsupervised Learning

• Task: Drawing inferences from input data without labeled responses.
• Process: Classification or categorization is not included in the observations. There are no output values which means there is no estimation of functions
• Common Method: cluster analysis, used for exploratory data analysis to find hidden patterns or grouping in data.

Reinforcement Learning

• Task: Maximizing rewards by getting an agent to act in the world.
• Process: Discover which actions yield the most reward. Actions may affect not only the immediate reward but also situations and, through that, all subsequent rewards.
• Method: Teach a dog a new trick. You cannot tell it what to do, but you can reward/punish it if it does the right/wrong thing. It has to find out what it did that made it get the reward/punishment. A similar method is used to train computers to do many tasks, such as playing backgammon or chess, scheduling jobs, and controlling robot limbs.
• Reinforcement learning teaches from trial and error, differing from supervised learning's expert-provided examples.