Untitled Quiz

Introduction to Machine Learning

• Machine Learning (ML) is a branch of AI/computer science aiming to develop algorithms capable of learning and improving from experience.
• These algorithms analyze and interpret data, identifying patterns, making predictions, and making decisions based on training information.

Note

• These slides summarize class topics.
• They do not provide all necessary details.
• They are not the only study material.

Contents

• Introduction
• Machine learning processing steps
• Example
• Learning process
• Hard and soft computing
• Machine Learning evolution
• Conclusion
• Bibliography

1 | Introduction

• Machine learning (ML) is a branch of AI/computer science.
• ML aims to develop algorithms that learn and improve from experience.
• ML algorithms analyze data to identify patterns, make predictions, and make decisions.
• These algorithms improve their performance over time.

1 | Introduction

• ML aims to solve problems without explicit programming for every task.
• Data analysis and pattern recognition are central to ML.

1 | Introduction (continued)

• ML algorithms focus on improving performance metrics such as sensitivity, specificity, and accuracy.
• ML algorithms need training data to learn and make predictions or decisions.

1 | Introduction - ML vs. AI vs. DL

• AI: Mimicking human behavior with symbolic and data-based techniques
• ML: Subset of AI, using data to improve with experience
• DL: Subset of ML, enabling multi-layer neural networks

1 | Introduction - ML roots/applications

• ML has roots in AI and computer science, focusing on data-driven learning and improving performance from data.
• Focuses on improving the performance of learning methods.
• ML can process massive amounts of data in a variety of contexts.

1 | Introduction - ML models/techniques

• Clustering
• Neural Networks
• Decision Trees
• Fuzzy Systems
• Deep learning methods

1 | Introduction - Statistics vs. Machine Learning

• Statistics: Based on mathematics, focuses on inference from sample sets, and hypothesis testing
• Machine Learning: Based on AI, focuses on optimization problems, such as improving the performance of learning.
• Statistics has more theoretical foundations, while ML is often more practical and heuristic.

1 | Introduction - ML data types

• Static data: Values independent of each other (e.g., height/weight)
• Temporal data: Data sequences dependent on past values (e.g., time series).

1 | Introduction - ML problems

• Describing data
• Classifying data
• Regression
• Time-series data
• Predict future values

2 | ML processing steps

• Data selection
• Preprocessing
• Data transform
• Modelling
• Evaluation

2 | ML processing steps (Data selection)

• Available data?
• Subset selection (privacy concerns?)
• Relevant characteristics/features?

2 | ML processing steps (Data types)

• Binary data (yes/no)
• Categorical data (nominal/ordinal)
• Continuous data (real values)

2 | ML processing steps (Data types – cont.)

• Static Data
• Temporal Data

2 | ML processing steps (Preprocessing)

• Data cleaning/preparation, crucial for quality analysis
• Address missing values and abnormal values (outliers)
• Noise reduction

2 | ML processing steps (Preprocessing - Outliers)

• Outliers: Values deviating from typical data.
• Examples: telemonitoring data, sales revenue data (and others).

2 | ML processing steps (Preprocessing - Noise)

• High-frequency and low-frequency noise can degrade data.
• Example: ECG data, with noisy signals.

2 | ML processing steps (Preprocessing - Missing Values)

• Missing data in datasets is common.
• Missing values should be replaced or removed to train models.

2 | ML processing steps (Data transform)

• Feature Extraction: Summarizing complex objects into simpler features.
• Feature engineering: Creating new features/attributes from existing ones.

2 | ML processing steps (Data transform – cont.)

• Data reduction: Reduction of higher dimension data
• Data redundancy elimination

2 | ML processing steps (Data transform – cont.2)

• Example: time series data
• Frequency and period are sufficient representations of a sine wave.

2 | ML processing steps (Data transform – cont.3)

• Example: ECG data
• ST deviation and PR interval are good parameters

2 | ML processing steps - Data transform (Redundancy)

• Identifying/removing redundant attributes
• Example: Using height, weight, BMI. BMI is a function of the other two.

2 | ML processing steps - Modelling

• Building models that solve specific problems via available features/attributes.

2 | ML processing steps - Modelling (cont'd)

• Models types: Linear / Non-Linear, Supervised / Unsupervised, Clustering / Classification / Regression / Prediction

2 | ML processing steps - Modelling (cont'd.2)

• Linear Models
• Non-Linear Models (example: Neural networks)

2 | ML processing steps - Evaluation/Validation

• Model quality assessment methods
• Methods to check model reliability: Sensitivity / Specificity

2 | ML processing steps - Evaluation/Validation (cont'd)

• Error calculation example: Prediction error to determine model fitness.
• Accuracy metrics: Accuracy, Precision, Recall, F1-Score
• SSE, R-squared values in regression.

3 | Example: Diagnosing Diabetes

• Classification problem (binary)
• Target outcomes (diabetes/no diabetes)
• Data selection: Clinical patient data (age, weight, gender..)
• Data transformation: summarizing data, computing statistical information (mean, standard deviation, etc).

3 | Example: Diagnosing Diabetes (cont'd)

• Preprocessing: Handling missing values, outliers.
• Input features: Summarized statistical information
• Output predictions (Yes/ No Diabetes)
• Model training: Building a model.

4 | Learning Process

• Inductive Learning Hypothesis: Models well on training data -> likely to generalize to new data.
• Generalization: Models ability to perform on unseen data.
• Evaluation: Assessing quality

4 | Learning Process - Evaluation

• Hold-out validation: Splitting the data into training and testing sets to evaluate generalizability.
• Cross-validation (k-fold): Repeated hold-out validation improving robustness.

4 | Learning Process - Model Structure & Learning Process

• Model's structure (e.g., parameters) are assumed for the process
• Learning algorithms infer or compute a set of parameters from the training dataset.

4 | Learning Process - Model Structure & Learning Process (continued)

• Ex. Autoregressive model: Compute coefficients for time series prediction.
• Ex. Neural network: Compute neuronal network weights and biases.

4 | Learning Process - Data Characteristics

• Data balance: Training data should include representative instances for all classes (positive and negative examples).

4 | Learning Process - Data Characteristics (continued)

• Representative training data: Includes all possible combinations of input attributes from which new data instances may appear

5 | Hard and Soft Computing

• Hard computing: Based on mathematical and exact equations
• Soft computing: Tolerates imprecision and uncertainty in real-world applications
• Example: Automatic piloting for cars

5 | Hard and Soft Computing (continued)

• Biological inspiration: Artificial neurons inspired by biological neurons
• Fuzzy logic is an example of soft computing (fuzzy brain). Neural Network model inspiration as well

6 | Machine Learning Evolution

• Overview of the historical development of AI and Machine learning, including notable models/algorithms from various epochs
• Different phases of AI development (First/Second/Third Golden Ages)
• Key breakthroughs such as the neural network, Linear/non-linear Models etc

6 | Machine Learning Evolution (continued)

• Connectionist models (e.g., neural networks) and their ability to approximate non-linear mappings.
• Recurrent Neural Networks(RNNs), long short-term memory (LSTM).

6 | Machine Learning Evolution (continued, Generative AI)

• Generative AI: Creating new content based on learned patterns from large datasets
• Large Language Models (LLMs) and transformers
• Transformer models: Processing correlated word sequences without the need for recurrent structures.
• GPT-3, GPT-4

7 | Conclusion

• ML has various algorithms (no single best for each problem).
• Success in ML relies on experience, trial and error, and scientific principles
• Generalization capability is crucial
• ML is based on data and mathematical principles.

Bibliography

• A list of referenced sources, including books and webpages, are cited for further reading.