te.pdf

# Adaptive Machine Learning - **Concept**: Methods that adapt models in response to new data or contextual changes - **Importance**: Addresses the challenge of evolving data sources - **Focus**: Interpreting data based on its dynamic nature - **Key Element**: Incorporates time into the data abstra...

# Adaptive Machine Learning - **Concept**: Methods that adapt models in response to new data or contextual changes - **Importance**: Addresses the challenge of evolving data sources - **Focus**: Interpreting data based on its dynamic nature - **Key Element**: Incorporates time into the data abstraction through data streams ## Examples of Adaptive Data - Sensor Data (IoT) - Video, Audio, Camera Feeds - Network Traffic ## What are Data Streams? - Sequences of instances, possibly infinite, each item having a timestamp - Temporal order is crucial - **Stream Learning**: Machine learning methods designed to build and maintain models in real-time (or near real-time) while instances arrive one by one. - Objective: Understand patterns and predict future items based on the continuous flow of data ## ML for Batch (Static) Data vs ML for Streaming (Online) Data ### Batch ML - **Characteristics**: - Fixed-size dataset - Random access to any instance - Well-defined phases (train, validation, test) - **Challenges**: - Noise - Missing data - Imbalance - High dimensionality ### Stream ML - **Characteristics**: - Continuous flow of data - Limited time to inspect data points - Interleaved phases (train, validation, test) - **Concept Drift**: - The world is dynamic, changes occur all the time. - These changes affect our machine-learning models. - We need to: - Detect, understand and react to changes in the data. - Learn new concepts without forgetting old concepts. ## Concept Drift Examples: - **Learn to classify new classes** - **Update model to accommodate for changes within existing classes** - **Forget that which is no longer needed** ### Related Research Areas/Jargon - **Class Evolution (Stream Learning)** - **Class Incremental (Continual Learning)** - **Concept Drift (Stream Learning)** - **Domain Incremental (Continual Learning)** ## What If the Data Distribution Changes? - **Concept**: The data distribution may change overtime, leading to an underperforming model. - **Solution**: - **Detection Algorithms**: Identify changes in the data distribution. - **Model Update**: Adapt the model in response to detected changes. ### Key Questions: - What data should we use to train the updated model? - How do we detect changes? - What can the detection algorithm observe? ### Types of Distribution Changes: - **Real Drift**: - **Original Data**: The true underlying distribution of the data changes. - **Real Concept Drift**: Changes in the relationship between input variables (X) and the target variable (Y). - **Virtual Drift**: Changes in the distribution of the input variable (X) without affecting the relationship with Y. ## Abrupt and Gradual Drifts vs Their Impact on Accuracy - **Abrupt Drifts**: Sudden, significant changes in the data distribution. - **Gradual Drifts**: Slow, incremental changes over time. - **Reoccurring Concepts**: Temporary shifts in the data distribution that may reappear. - **Outliers**: Isolated data points that do not represent concept drift. ### The Design of a Change Detector: A Tradeoff - **True Changes Detection**: Identify real changes in the data. - **False Alarms Avoidance**: Minimize spurious detections. ### Categories of Change Detectors - **Memory-Based Detectors**: - **Sequential Analysis**: Store data from the stream (e.g., CUSUM, Page-Hinkley test). - **Memoryless Detectors**: - **Statistical Methods**: Use statistical techniques (e.g., DDM, EDDM). ### Window-Based Methods (ADWIN) - **Concept**: Use a sliding window to monitor changes in the distribution by comparing recent data with past data. - **Supervised Detection**: Monitored metrics include accuracy. - **Unsupervised Detection**: Focuses on detecting changes in the distribution of input variables (P(X)). ### Adaptive Window (ADWIN) - **Key Feature**: Uses adaptive windows of variable size, recalculated based on the observed change in the data. - **Purpose**: Increases stability when there is no change and adapts quickly when a change is detected. ## Adaptive Machine Learning: Can We Build a Method That Adapts Itself to Changes Automatically? - **Adaptive Random Forest (ARF)**: A streaming version of the original random forest algorithm by Breiman. - **Key Features**: - Uses a variation of the Hoeffding tree. - Includes a drift detector for each base model. - Background learners are activated when a drift is detected. - Forecasts are generated based on the 'best' learner at a given time. ### ARF: Adaptive Aspects - **Drift Detection**: Relies on the adaptive window (ADWIN) algorithm. - **Background Learners**: Started when a drift is detected, their feature space may differ from the foreground learner. - **Foreground Learner Replacement**: The background learner replaces the foreground learner after a drift is detected. ## Adaptive ML: Stream ML? - **Hybrids Exist**: A combination of batch and stream ML approaches can be used for specific tasks, focusing on training a batch model and then monitoring changes in real-time. - **Key Differences**: While both are related to time-sensitive data, streaming data is typically considered independent and identically distributed (i.i.d), whereas time series data may exhibit dependencies. - **Continual Learning vs Stream ML**: While similar, their goals differ. - **Continual Learning**: Focuses on preserving knowledge while adapting to new information. - **Stream ML**: Emphasizes plasticity and adapting to ever-changing data. ### Open Research Question: - **Concept Drift vs Anomaly Detection**: Differentiating between true concept drift and anomalous data remains a challenge. ## Uncertainty Quantification - **Why Evaluate Models?** To ensure models are fit for purpose and reliable for unseen data. - **Assessing Uncertainty**: Quantifying the model's confidence in its predictions. ### Good Uncertainty Quantification Methods: - **Calibration**: The predicted probabilities or uncertainty scores should align with the actual outcomes. - **Sharpness**: The confidence scores should reflect the concentration of the predictive distributions. ### Conformal Predictions: Classification - **Concept**: Provides prediction sets instead of single labels, offering a confidence level for each prediction. - **Guarantee**: The predicted set includes the true class with a specified confidence level. ### How Conformal Predictions Work: - **Training:** Train a classifier using the training data. - **Calibration:** Calculate the model's performance based on a separate calibration set to identify how well each class conforms. - **Prediction Set Creation**: Generate prediction sets for new instances based on the calibration scores. ### Pros of Conformal Predictions: - **Versatility**: Works well with any type of classifier. - **Nuanced Predictions**: Provides a more comprehensive prediction than a single label. ### Cons of Conformal Predictions: - **Data Requirement**: Requires a separate calibration set, reducing the training data. - **Computational Cost**: Can be slow for complex models. - **Large Prediction Sets**: Can reduce specificity, especially in cases of high uncertainty. ### Conformal Predictions for Regression: - **Prediction Intervals**: Instead of prediction sets, prediction intervals offer coverage guarantees. - **Concept**: Provide a range around the predicted value where the true value is likely to fall. ### Mean-Variance Estimation: - **Concept**: Generates prediction intervals by modeling the variance of the predicted outcomes. - **Purpose**: Provides a better understanding of the spread around the mean prediction. - **Steps**: - **Model the mean**: Predict the mean value. - **Estimate the variance**: Model the standard deviation. - **Construct prediction intervals**: Define a range within which the true value is likely to fall. ### Mean-Variance Estimation (MVE): A Powerful Tool for Uncertainty Quantification - **Widely Used**: In regression tasks to estimate both the predicted value (mean) and the associated uncertainty (variance). - **Extending Regression Algorithms**: MVE's additional layer quantifies uncertainty, making it applicable to various regression tasks. ### Evaluating Prediction Intervals: - **Calibration**: Measures how well the uncertainty scores align with the actual outcomes. - **Sharpness**: Indicates the concentration of the predictive distributions (e.g., interval width). ## Semi-Supervised Learning - **Concept**: Combines labelled and unlabeled data to train a model, leveraging the abundance of unlabeled data while minimizing the need for manual annotation. - **Challenge**: Minimizing the risk of overfitting while maximizing the value of unlabeled data. ### Active Learning: Querying the Most Informative Data Points - **Objective**: Guide the training process by requesting labels for the most uncertain instances. - **Goal**: Improve model accuracy by focusing on informative examples. - **Oracle**: A source of ground truth labels for queried data points. - **Limitations**: Requires a source of labels and can be computationally expensive. ### Semi-Supervised Learning: Bridging Supervised and Unsupervised Learning - **Concept**: Leverages unlabeled data without requiring manual labeling. - **Foundation**: Combines aspects of supervised and unsupervised learning. ### Applications: - **Classification**: Identifying the class of an instance. - **Clustering**: Grouping similar data points into clusters. ### Assumptions for Semi-Supervised Learning: - **Smoothness**: Samples that are close in the input space should have similar labels. - **Low-Density**: The decision boundary should not pass through regions with high data density. - **Manifold Assumption**: Data points on the same low-dimensional manifold should share the same label. ### Categories of Semi-Supervised Learning: - **Transductive Learning**: Focuses on generalizing to the given test set. - **Inductive Learning**: Aimed at generalizing beyond the data present. ### Semi-Supervised Learning Algorithms: - **Self-Training**: The model iteratively predicts labels for unlabeled data, adding the most confident predictions to the labeled set. - **Expectation-Maximization (EM)**: Alternates between assigning soft labels (E-step) and updating model parameters (M-step). - **Disagreement-Based Learning (Co-training)**: Multiple models are trained on different views of the data. Their disagreement aids in labeling and model improvement. - **S3VM (Semi-Supervised Support Vector Machines)**: Extends the support vector machine (SVM) framework by leveraging both labeled and unlabeled data to create a more robust decision boundary. - **Graph-Based SSL**: Constructs a graph where nodes represent data points and edges represent similarity. Labels are propagated from labeled to unlabeled points based on the graph structure. ### Self-Training: A Wrapper Method - **Concept**: Uses a classifier to predict labels for unlabeled data, adding the most confident predictions to the labeled set. - **Process**: - Train a classifier on the labeled data. - Predict labels for unlabeled data. - Add highly confident predictions to the labeled dataset. - Iterate until significant improvements are not observed. - **Advantages**: Simple and adaptable to various classifiers. - **Limitations**: Susceptible to bias propagation and overfitting, requiring careful stopping mechanisms. ### Learning by Disagreement - **Concept**: Leverages multiple classifiers trained on distinct views of the data to enhance learning. - **Objective**: Improve accuracy by capitalizing on the disagreement between models trained on different data or features. - **Multi-View Learning**: Extending learning by disagreement by adding more learners, encouraging a 'majority trains the minority' approach. - **Artificial View Creation**: Inducing multiple views using techniques like random subspaces. ## Unsupervised Learning - **Concept**: Discovering patterns, relationships, and structures within data without explicitly labeled information. - **Goal**: Explore the inherent organization of the data and identify hidden patterns or unknown relationships. ### Unsupervised Learning Techniques: - **Association Rule Mining**: Identify relationships or dependencies between items in a dataset. - **Anomaly Detection**: Identify data points that significantly differ from the rest of the dataset. - **Generative Models**: Learn the underlying data distribution to generate new samples. - **Dimensionality Reduction**: Reduce the number of features while preserving essential information. ### Association Rule Mining: Uncovering Hidden Relationships - **Concept**: Discover if-then statements that identify relationships or dependencies. ### Key Elements: - **Support**: The proportion of transactions where the antecedent and consequent appear together. - **Confidence**: The likelihood of finding the consequent given the presence of the antecedent. ### Applications of Association Rule Mining: - **Recommendation Systems**: Suggest products based on past purchase history. - **Market Basket Analysis**: Identify products frequently purchased together. - **Diagnosis**: Identify potential health risks based on observed symptoms. ### Anomaly Detection: Identifying Outliers - **Challenge**: Detecting deviations from the expected data distribution. - **Key Aspect**: Distinguishing between anomalies and imbalanced classifications. - **Common Assumption**: Lack of readily available examples of the anomalous class. - **Importance**: Important for fraud detection, system monitoring, and data quality control. ###Types of Anomalies: - **Point (Global) Anomalies**: Individual data points that differ significantly from the majority of the dataset. - **Contextual (Conditional) Anomalies**: Data points that are normal in one context but anomalous in another. - **Collective Anomalies**: Groups of data points that together represent abnormal behavior, even if individual points may not be outliers. ### Anomaly Detection Algorithms: - **Local Outlier Factor (LOF)**: Measures the local density of a point to identify those with significantly lower density than their neighbours. - **One-Class SVM**: Learns a decision boundary around the normal data and classifies points outside the boundary as anomalies. - **Isolation Forest**: An ensemble-based method that isolates anomalies by randomly splitting features. - **DBSCAN**: A density-based clustering algorithm that identifies outliers as points that do not belong to dense regions. - **Autoencoders**: Neural networks that reconstruct input data. Points with high reconstruction errors are considered anomalies. ### Isolation Forest: A Powerful Anomaly Detection Algorithm - **Concept**: Uses decision trees to isolate anomalies, which are typically easier to isolate than normal points. - **Key Feature**: Anomalies appear on shorter branches, while normal points go deeper in the tree. - **Applications**: Detecting global anomalies in unsupervised settings. ### Key Hyperparameters for Isolation Forest: - **n_estimators**: Number of trees in the forest. - **max_samples**: Number of samples used for training each tree. - **max_features**: Number of features used for splitting data in each tree. - **bootstrap**: Sample with replacement (default: False). - **contamination**: Proportion of anomalies in the dataset. ### Contamination: Defining the Anomaly Threshold - **Concept**: Specifies the proportion of data points that are considered anomalies. - **Impact**: Influences the algorithm's sensitivity to detecting anomalies, with higher contamination leading to more lenient detection. ### Best Practices for Using Isolation Forest: - **Known Outliers**: If the proportion of outliers is known, explicitly set the contamination parameter for optimal results. - **Uncertain Outliers**: Test different contamination values to find the optimal balance between true anomalies and false positives. ## Unsupervised Learning x Supervised Learning - **Supervised Learning**: Trains a model on explicitly labeled data to make predictions on unseen data. - **Unsupervised Learning**: Focuses on discovering patterns and relationships within data without explicit labels. ## Clustering: Grouping Similar Data Points - **Concept**: Grouping data points based on their features, minimizing the distance between points within a cluster while maximizing the distance between clusters. - **Objective**: Discover the inherent structure of the data, identify relationships, and uncover hidden subgroups. - **Applications**: Customer segmentation, anomaly detection, and image segmentation. ### Key Considerations for Choosing a Clustering Algorithm: - **Goal**: The desired outcome of the clustering process. - **Evaluation Metric**: The criterion used to assess the quality of the clustering results. - **Number of Clusters**: The desired number of clusters. - **Cluster Shape**: The expected shape of the clusters (e.g., spherical or non-spherical). ### Clustering Algorithms: - **K-Means**: A centroid-based algorithm that partitions data into k clusters, minimizing the distance between each point and its assigned centroid. - **DBSCAN**: A density-based algorithm that searches for dense regions in the data space, identifying clusters of varying shapes and densities. ### K-Means: Centroid-Based Clustering - **Concept**: Assigns each data point to the closest centroid, iteratively updating the centroids until convergence. - **Hyperparameter**: K (the number of clusters). - **Limitations**: Sensitive to initialization and can produce spherical clusters. ### DBSCAN: Density-Based Clustering - **Concept**: Identifies core points (data points with a minimum number of neighbours within a specified radius) and expands clusters based on the connectivity of core points. - **Key Parameters**: - **MinPts**: Minimum number of points required to form a cluster. - **ε**: Radius for defining the neighborhood of each point. - **Advantages**: Handles clusters of varying shapes, sizes, and densities, and is robust to outliers. ### Elbow Method: Determining the Optimal Number of Clusters - **Concept**: Uses a clustering quality measure to assess the performance of different clusterings with varying numbers of clusters. - **Objective**: Identify the 'elbow point' on the plot, representing the optimal number of clusters. - **Measure**: Commonly used measures include the Within-Cluster Sum of Squares (WCSS). ## Ensembles: Combining Multiple Learners for Improved Performance - **Concept**: Combining multiple 'weak' learners to create a stronger, more robust predictor. - **Advantages**: - **Reduced Overfitting**: Averaging predictions across multiple learners. - **Increased Robustness**: More diverse models lead to greater robustness. ### Bagging: Bootstrap Aggregating - **Concept**: Trains multiple models on different bootstrap samples of the original dataset. - **Key Feature**: Each bootstrap sample contains instances drawn with replacement, leading to a diverse set of models. - **Prediction Aggregation**: Predictions from the individual models are combined using majority vote. #### Bagging: Local vs. Global Randomization - **Local Randomization**: Sampling instances and features randomly for each tree (e.g., random forest). - **Global Randomization**: Dividing the dataset into subspaces and training models on different subspaces (e.g., random subspaces). ### Boosting: Sequential Ensemble Learning - **Concept**: Trains multiple models sequentially, each model focusing on correcting the errors made by previous models. - **Objective**: Build a strong ensemble by iteratively focusing on difficult instances. - **Boosting Algorithms**: - **AdaBoost**: Adjusts the weights of instances based on their classification errors, giving more weight to misclassified instances. - **Gradient Boosting Machines (GBM)**: Trains models sequentially to minimize the residual errors of previous models' predictions. ## Gradient Boosting Machines (GBM): A Powerful Boosting Algorithm - **Concept**: Sequentially trains models to minimize the residuals of previous models' predictions. - **Key Feature**: Focuses on correcting residual errors, leading to more accurate and robust predictions. - **Regularization Techniques**: Enhances model generalization by introducing techniques like tree pruning and leaf-wise growth. - **Popular GBM Algorithms**: - **XGBoost**: Regularization, tree pruning, and parallel and distributed computing. - **LightGBM**: Leaf-wise growth, scalability, and efficient handling of categorical features. - **CatBoost**: Handles categorical features efficiently, enabling faster training. ## Model Selection - **Concept**: Choosing the best candidate model or set of hyperparameters from a pool of options. - **Strategies**: - **Static Selection**: Perform selection once after training, and the selected model is used for all future predictions. - **Dynamic Selection**: Select the model adaptively based on new data, changing patterns, or performance feedback. ### Model Selection Approaches: - **Grid Search**: Evaluate a fixed set of hyperparameters. - **Multi-Armed Bandit (MAB) Algorithms**: Explore a set of models' performances and dynamically adjust the selection based on observed results. - **Thompson Sampling**: A common MAB algorithm for exploring a set of models with uncertain rewards. ## Stacking: Hierarchical Ensemble Learning - **Concept**: Trains a meta-model to combine the predictions of multiple base learners. - **Objective**: Enhance performance by leveraging complementary strengths of base models. - **Key Features**: - Base models are often heterogeneous (e.g., decision trees, neural networks). - The meta-model learns the optimal way to integrate the predictions of the base models. ## Policy Evaluation - **Context**: Evaluating the performance of an agent interacting with an environment. - **Objective**: Measure how well the agent achieves its goals under a specific policy. - **Value Function**: Represents the expected reward accumulated by the agent from a specific state. ### Key Concepts in Policy Evaluation: - **State**: Represents the current situation of the agent within the environment. - **Action**: A decision made by the agent. - **Reward**: The numerical value received by the agent for performing an action. - **Policy**: A rule that specifies the actions the agent should take in each state. - **Discount Factor (γ)**: A value between 0 and 1 that weighs the importance of future rewards relative to immediate rewards. - **Expected Return**: The average total reward accumulated by the agent following a specific policy from a given state. ### Value Iteration: Calculating the Optimal Value Function - **Concept**: Iteratively updates the value function for all states until convergence, ensuring the optimal action is taken to maximize the expected return. - **Purpose**: Determine the best policy for the agent. ### Policy Iteration: Finding the Best Policy - **Concept**: Iterates between policy evaluation and policy improvement, finding the optimal policy that maximizes the expected return for all states. - **Steps**: - **Policy Evaluation**: Calculate the expected return for each state under a given policy. - **Policy Improvement**: Update the policy by selecting the action that maximizes the expected return. - **Convergence**: Stop the iteration process when no further improvements are possible, resulting in the optimal policy for the given MDP. ## Reinforcement Learning - **Concept**: An agent learns to interact with an environment by taking actions based on the received rewards. - **Objective**: Find a policy that maximizes the long-term cumulative reward. - **Key Concepts**: - **Agent**: The learner that interacts with the environment. - **Environment**: Everything external to the agent. - **State**: The current configuration or situation of the agent. - **Action**: The choices the agent can make within the environment. - **Reward**: The numerical value the agent receives for taking actions. - **Policy**: A rule that maps states to actions. ### Reinforcement Learning: A Framework for Decision-Making in Uncertain Environments - **Applications**: Robotics, game playing, finance, and healthcare. - **Key Challenge**: Handling uncertainty and managing the tradeoff between exploring new actions and exploiting known rewards. ### Elements of Reinforcement Learning: - **Markov Decision Processes (MDPs)**: Mathematical frameworks for formalizing reinforcement learning problems. - **State Transitions**: The probability of transitioning between states based on the agent's actions. - **Rewards**: The numerical values representing the desirability of different states. ### Planning in Reinforcement Learning - **Concept**: Pre-computing optimal policies by exploring the entire state space. - **Assumption**: The environment is known and deterministic. - **Applications**: Solving problems with a limited number of states and actions. ## Evolutionary Machine Learning - **Concept**: Applies evolutionary algorithms to optimize machine learning models, allowing for the simultaneous evolution of model structure and parameters. - **Objective**: Find optimal model structures and parameters by mimicking the processes of natural selection and evolution. - **Key Applications**: - **Model Selection**: Optimizing the architecture of neural networks. - **Feature Engineering**: Discovering and selecting the most informative features for a particular task. - **Hyperparameter Tuning**: Finding the best hyperparameters for a specific algorithm. ## Genetic Programming: Evolutionary Optimization for Model Structure - **Concept**: Evolves a population of tree-based models, evaluating them based on a fitness function and selecting the 'best' individuals for reproduction. - **Key Elements**: - **Terminals**: Represent variables, constants, or features. - **Functions**: Perform operations on terminals. - **Fitness Function**: Measures the performance of each individual in the population based on a specific objective. - **Genetic Operators**: Crossover, mutation, and reproduction are applied to generate new individuals. ### Genetic Programming Framework - **Initialization**: Create an initial population by randomly generating trees using terminals and functions. - **Evaluation**: Assess the fitness of each individual. - **Selection**: Select individuals based on their fitness for reproduction. - **Crossover**: Combine genetic material from two individuals by exchanging subtrees. - **Mutation**: Randomly modify a tree by changing terminals, functions, or parameters. - **Reproduction**: Create new individuals by replicating the best individuals in the population. ### Fitness Evaluation: Evaluating the Performance of Models - **Concept**: A function that measures the quality of a model based on a specific goal. - **Objective**: Guide the evolution process by promoting individuals with higher fitness. ### Crossover in Genetic Programming: Exchanging Genetic Material - **Concept**: Combines two parent trees to create a new individual by swapping subtrees. - **Purpose**: Explore new combinations of features and operations, improving model diversity. ### Mutation in Genetic Programming: Introducing Random Variation - **Concept**: Randomly modifies a tree by replacing subtrees, changing terminals, or altering function parameters. - **Purpose**: Introduce new variations into the population and mitigate the risk of local optima. ### Parameters to Consider for Genetic Programming: - **Population Size**: The number of individuals in the population. - **Number of Generations**: The number of iterations of the evolutionary process. - **Maximal Tree Depth**: The maximum depth of trees in the population. - **Crossover Rate**: The probability of applying crossover during reproduction. - **Mutation Rate**: The probability of applying mutation during reproduction. - **Reproduction Rate**: The probability of replicating existing individuals. - **Tournament Selection Size**: The number of individuals in a tournament to select the best parent. ### Applying Genetic Programming to Classification - **Concept**: Develops a model to predict the class of an instance. - **Process**: - **Evolution**: Evolves a population of tree-based models. - **Classification**: Convert the output to a class label by defining a threshold based on the real-value output. ### Advanced Topics in Genetic Programming: - **Learning Coefficients**: Optimizing coefficients within the program tree alongside model structure. - **Data Types**: Handling different data types (e.g., floating numbers, boolean values). - **Functions**: Introducing functions that operate on mixtures of data types. ## Evolutionary Neural Network Architecture Design - **Concept**: Apply evolutionary algorithms to design neural network architectures, automating architecture search and enhancing model performance. ### Design Considerations for Neural Networks: - **Number of Hidden Layers**: The depth of the network. - **Number of Hidden Nodes**: The width of the network. - **Connectivity Structure**: How nodes are connected (e.g., fully connected or skip connections). - **Activation Functions**: The choice of activation functions. - **Regularization Techniques**: Techniques to prevent overfitting (e.g., dropout, L1/L2 regularization). ## Encoding and Decoding Neural Network Architectures - **Encoding**: Represent neural network architectures as a series of parameters that can be manipulated by genetic operators. - **Decoding**: Convert the encoded representation back into a neural network architecture for training and evaluation. ### Evaluating Neural Network Architectures - **Backpropagation and Gradient Descent**: Train the network using standard optimization algorithms. - **Performance Metrics**: Evaluate the performance of the network based on the task at hand. ## Probabilistic Inference - **Concept**: The process of inferring the probability distribution over hidden variables, given observed data. - **Objective**: Estimate the most likely values of hidden variables, taking into account the uncertainty associated with the observed data. ### Key Concepts in Probabilistic Inference: - **Prior Probability**: The initial belief about the probabilities of events. - **Likelihood**: The probability of observing the data given a specific value of the hidden variable. - **Posterior Probability**: The updated belief about the probabilities of events, after considering the observed data. ### Bayes' Theorem: Updating Probabilities with New Evidence - **Concept**: A mathematical formula that relates prior, likelihood, and posterior probabilities. - **Purpose**: Update the probability of a hypothesis given new evidence. ### Maximum A Posteriori (MAP): Finding the Most Likely Value of the Hidden Variable - **Concept**: Estimate the most likely value of the hidden variable by maximizing the posterior probability. - **Key Feature**: Balances the influence of the prior probability and the likelihood. ### Laplace Smoothing: Addressing Overfitting - **Concept**: Adding a 'pseudo-count' of 1 to each category in order to prevent zero probabilities. - **Purpose**: Improve the accuracy of estimated probabilities, especially when the amount of data is limited. ### Entropy: Measuring Uncertainty - **Concept**: A measure of uncertainty or randomness associated with a probability distribution. - **Objective**: Quantify the amount of information needed to reduce uncertainty. ### Conditional Independence: Simplifying Probabilistic Models - **Concept**: The assumption that two variables are independent, given the value of a third variable. - **Benefit**: Reduces the complexity of probabilistic relationships and makes it easier to model complex systems. ## Factorization: Breaking Down Complex Joint Distributions - **Concept**: Expressing a joint probability distribution as a product of simpler conditional distributions. - **Purpose**: Simplify the calculation of probabilities and reduce the complexity of probabilistic models. - **Key Feature**: The product rule: P(X,Y) = P(X)P(Y|X). ### Bayes' Nets: Representing Probabilistic Relationships - **Concept**: Directed acyclic graphs (DAGs) that visually represent the relationships between variables. - **Key Features**: - Nodes represent variables. - Edges represent conditional dependencies between variables. - **Applications**: Modeling complex systems in medicine, engineering, and finance. ## Inference in Bayes' Nets - **Concept**: The process of using Bayes' theorem and the structure of a Bayes' net to calculate conditional probabilities. - **Key Aspects**: - The choice of evidence (observed variables). - The structure of the Bayes' net. ### The Power of Inference in Bayes' Nets: - **Reasoning about Causes from Their Effects**: Inferring the cause of an event given the observed effects. - **Scaling Up**: Handling more complex models with a larger number of variables. ## Two Ways to Express Independence - **Joint Probability**: Two variables are independent if their joint distribution is equal to the product of their marginal distributions. - **Conditional Probability**: Two variables are independent if their conditional probability, given a third variable, is equal to the marginal probability of the first variable. ## Maximum Likelihood Estimation (MLE): Finding the Best Parameters for Categorical Distributions - **Concept**: Estimating parameters of a categorical distribution by maximizing the likelihood of observing the given data. - **Objective**: Find the distribution that best explains the observed data. ### Overfitting and Smoothing: Addressing Data Scarcity - **Concept**: The problem of overfitting when the data is limited in number of observations. - **Solution**: Laplace smoothing, which adds a 'pseudo-count' of 1 to each category to ensure all probabilities are greater than zero. ## Entropy of a Categorical Distribution: Measuring Uncertainty - **Concept**: Quantifies the uncertainty or randomness associated with a probability distribution over categorical outcomes. - **Objective**: Understand the amount of information needed to resolve uncertainty. ## The Importance of Probabilities in AI - **Uncertainty**: The foundation of AI systems. - **Decision-Making**: Incorporating probabilities into decision-making models to account for uncertainty. - **Learning**: Estimating probabilities from data to improve model accuracy. ## The Product and Sum Rules: The Cornerstones of Probability Theory ### Product Rule: - **Concept**: Relates joint probabilities to conditional probabilities. - **Formula**: P(X,Y) = P(X)P(Y|X). - **Applications**: - Calculating the probability of a joint event. - Inferring conditional probabilities given joint probabilities. ### Sum Rule: - **Concept**: Calculates the probability of an event by summing the probabilities of all its possible outcomes. - **Formula**: P(X) = ΣYP(X,Y). - **Applications**: - Calculating the marginal probability of a variable. - Deriving conditional probabilities. ## Linear Regression: A Powerful Tool for Predicting Continuous Variables - **Concept**: Model the relationship between a dependent variable (Y) and independent variables (X) using a linear equation. ### Key Components: - **Model**: Y = XW + ε, where W is the weight matrix and ε represents noise or error - **Objective**: Estimate the weight matrix W that minimizes the difference between predictions and observed values. ### Algorithms for Linear Regression: - **Gradient Descent**: Iteratively updates the weight matrix in the direction that minimizes the loss function. - **Pseudo-Inverse**: Directly solves for the weight matrix, requiring matrix inversion. ### Matrix Multiplication: A Fundamental Operation in Linear Algebra - **Concept**: Combining rows and columns of matrices to produce a new matrix. - **Applications**: Linear regression, least squares regression, image processing, and machine learning. ### Vectors in High Dimensions: Challenges of High Dimensionality - **Concept**: The curse of dimensionality refers to the challenges associated with dealing with data in high-dimensional spaces. - **Issues**: - Increased data sparsity. - Difficulty in defining distances between points. - Challenges in finding meaningful relationships between variables. ### Addressing the Curse of Dimensionality: - **Dimensionality Reduction**: Reducing the number of features while preserving relevant information. - **Feature Engineering**: Transforming features to improve model performance. ### Linear Regression: How It's Actually Done - **Gradient Descent**: An iterative optimization algorithm that minimizes the loss function by taking steps in the direction of the negative gradient. - **Pseudo-Inverse**: A direct method for solving linear equations, involving matrix inversion. - **Probability Theory**: Utilizing probabilistic models to infer the relationship between variables. ### The Gradient: A Powerful Tool for Optimization - **Concept**: A vector of partial derivatives that points in the direction of the steepest ascent. - **Applications**: - Finding the minimum or maximum of a function. - Training machine learning models. ### Chain Rule: Composing Derivatives for Multi-Variable Functions - **Concept**: A rule for calculating the derivative of a composite function. - **Purpose**: Allows for the calculation of complex derivatives by breaking them down into simpler derivatives. ### Rotation: Transforming Vectors in Two Dimensions - **Concept**: A process of transforming a vector by changing its orientation but preserving its length. - **Key Element**: Rotation matrices, which transform vectors using trigonometric operations. - **Orthogonal Matrices**: Matrices that preserve the length and angle of vectors. ### Inverse of a Matrix: Undoing Transformations - **Concept**: A matrix that, when multiplied by the original matrix, results in the identity matrix. - **Applications**: - Solving linear equations. - Inverting transformations. ### Singular Value Decomposition (SVD):