Introduction to Machine Learning

Machine Learning

• Machine Learning (ML) is the science of programming computers to learn from data
• It allows computers to learn without being explicitly programmed

Objectives of Machine Learning

• Understand the definition of Machine Learning
• Understand why to use Machine Learning
• Learn different types of Machine Learning systems

Defining Machine Learning

• ML is the process of programming computers so they can learn from data
• ML provides computers the ability to learn without explicit programming
• A computer program learns if its performance improves with experience, according to a specific task and performance measure

Why Use Machine Learning?

• ML simplifies complex problems like spam filtering, which require hand-tuning using traditional programming
• Traditional programming methods involve:
  • Analyzing spam patterns
  • Writing detection algorithms
  • Testing and repeating the process
• ML enables automatic adaptation to changes
• ML algorithms automatically learn to identify complex patterns
• ML can provide insights into complex problems and large amounts of data
• Traditional approaches require manually writing rules, while ML can automate this effectively

Types of Machine Learning Systems

• Systems can be classified into broader categories based on:
  • Training with human supervision (supervised, unsupervised, semi-supervised, reinforcement learning)
  • Incremental learning (online vs. batch learning)
  • Generalization approach (instance-based vs. model-based learning)

Supervised Learning

• Algorithms are trained with labeled data
• Enables the algorithm to learn from examples
• A common task is classification, like spam filtering
• Another task is to predict a target numeric value (regression)

Supervised Learning Algorithms

• Includes k-Nearest Neighbors
• Includes Linear Regression
• Includes Logistic Regression
• Includes Support Vector Machines (SVMs)
• Includes Decision Trees and Random Forests
• Includes Neural Networks

Unsupervised Learning

• Training data is unlabeled
• Algorithms learn without a teacher

Types of Unsupervised Learning

• Includes Clustering algorithms such as:
  • K-Means
  • DBSCAN
  • Hierarchical Cluster Analysis (HCA)
• Includes Anomaly detection such as:
  • One-class SVM
  • Isolation Forest

Semi-Supervised Learning

• Deals with partially labeled training data using mostly unlabeled data and a bit of labeled data

Reinforcement Learning

• The system (agent) observes the environment
• The system selects and performs actions
• The system receives rewards or penalties, learning the best strategy (policy) to maximize rewards over time

Batch and Online Learning

• Classifies ML systems by their learning incrementality from incoming data

Batch Learning

• The system is incapable of learning incrementally
• The systems is trained using all available data, usually offline due to high computational cost
• The system is then launched and applies what it has learned

Online Learning

• The system is trained incrementally with sequential data instances through mini-batches
• Each learning step is fast and cheap, allowing the system to learn new data as it arrives

Instance-Based vs Model-Based Learning

• Systems are categorized by how they generalize new instances
• Most Machine Learning tasks are about making predictions
• Instance-based learning learns examples by heart and generalizes by comparing new instances to learned ones
• Model-based learning builds a model from a set of examples and uses the model to make predictions

Main Challenges of Machine Learning

• Selecting a learning algorithm and training it on some data, can lead to “bad algorithm” and “bad data”
• Insufficient Quantity of Training Data
• Nonrepresentative Training Data
• Poor Quality Data
• Irrelevant Features
• Overfitting the Training Data
• Underfitting the Training Data
• Stepping Back
• Testing and Validating