Machine Learning Lecture Summaries

Combining multiple models to make a more robust model

A type of ensemble model consisting of multiple decision trees

A technique used in Random Forests to create multiple bootstrap datasets by randomly sampling with replacement

A technique used in Random Forests to randomly select features for splits in decision trees, adding diversity

Parameter that controls the number of decision trees in a Random Forest

Parameter that controls the maximum number of features used in a split of a decision tree in a Random Forest

Parameter that controls the maximum depth of a decision tree in a Random Forest

Parameter that controls the minimum number of samples required to split a node in a decision tree in a Random Forest

A model that fits training data too closely and performs poorly on new data due to memorizing training examples instead of learning general patterns.

A model that is too simple and fails to capture important patterns in the data, leading to poor performance on both training and new data.

A fundamental concept in machine learning that involves balancing the trade-off between bias (due to simplifying assumptions) and variance (due to sensitivity to data variations) to achieve optimal model performance.

A technique used to improve the generalizability of a machine learning model by splitting the data into training and testing sets. This allows the model to learn patterns from the training data and be evaluated on unseen data from the testing set.

A type of machine learning model that uses a tree-like structure to represent decisions and their possible consequences. It predicts a target variable by following a series of rules based on the values of input features.

A measure of how well a regression model fits the data. A value closer to 1 indicates a better fit, while a value closer to 0 indicates a poor fit.

A statistical measure that calculates the average absolute difference between the predicted values and the actual values. It provides an idea of how far off the model's predictions are from the real outcomes.

A statistical measure that calculates the square root of the average squared differences between the predicted values and the actual values. It penalizes larger errors more than MAE and provides a more accurate measure of the overall prediction error.

A technique using multiple decision trees where each tree learns from the errors of the previous ones, improving prediction accuracy.

Representing discrete data, like words or images, in a continuous vector space, allowing for easier analysis and comparison based on distances.

The relationship between two events where one directly causes the other, holding all other factors constant.

The ability of a model to accurately predict new data that was not seen during training

A parameter that controls how much each tree in GBTs contributes to the final prediction.

Artificial Neural Networks (ANNs) are a type of machine learning model that are particularly powerful in capturing complex patterns and relationships in data. They are often used for both classification (predicting categories) and regression (predicting continuous values).

An ANN's structure is comprised of several layers. The input layer receives the data, which is then processed through one or more hidden layers, ending with the output layer which produces the prediction. These layers are connected by weights and biases that determine the flow and strength of signals.

ANNs with just one hidden layer are considered 'shallow,' while those with two or more layers are classified as 'deep.' The depth of an ANN can allow it to learn more intricate and complex patterns.

Training an ANN involves finding the ideal set of weights and biases that minimize the difference between the predicted output and the actual target values. This process is called 'backpropagation' and it uses optimization algorithms like gradient descent to learn efficiently.

The 'loss function' quantifies the error between the predicted and actual values. Popular choices include Mean Squared Error (MSE) for regression and Cross-entropy (log-loss) for classification. It guides the training process by indicating how far off the predictions are.

Hyperparameters are settings that are not learned directly from the data, but instead are set beforehand. Important examples include the number of hidden layers and nodes, the batch size (how much data is used per iteration), and the learning rate (controlling optimization steps).

Hyperparameter tuning involves systematically experimenting with different values for these hyperparameters to find the combination that leads to the best performance on the unseen data. This process can be aided by tools like GridSearchCV, which automates the search for optimal values.

K-Fold Cross Validation is a technique used to assess the generalization performance of an ANN. It involves dividing the data into K folds and iteratively training the model on K-1 folds and evaluating its performance on the remaining fold. This process is repeated K times, using each fold as the test set once.

AI systems should be developed and deployed responsibly, considering ethical implications like privacy, discrimination, and environmental impact.

AI systems can unintentionally reinforce existing biases and stereotypes if training data is not representative of the real world.

AI can pose risks to privacy by collecting, storing, and using sensitive personal information without proper safeguards.

Techniques that make AI models more transparent and understandable to humans, helping to explain their decision-making processes.

AI can play a crucial role in addressing climate change by optimizing energy consumption, improving transportation efficiency, and supporting climate policy analysis.

AI systems should consider the specific context and domain-specific needs when generating explanations, ensuring they are tailored to the audience.

Balancing the desire for transparent AI explanations with maintaining the model's performance.

Regulations and standards are needed to ensure responsible development and deployment of AI systems.

A method for understanding the influence of features on a model prediction; it assesses how variations in input affect the output.

Measures how variations in model inputs influence the output. It determines which features have the most significant impact on the model's predictions.

Measures how predictions change when a feature's values are randomly shuffled. This technique reveals the importance of features by observing the impact of disrupting their information.

Visual aids like ICE (Individual Conditional Expectation) plots and PDP (Partial Dependence Plots) that clarify model decisions. They provide insights into how feature values affect the model's predictions across individuals or groups.

A method for explaining model predictions that focuses on the contribution of individual features to the overall output. It aims to assign a value to each feature representing its impact on the model's prediction for a specific instance.

A method for explaining model predictions that focuses on local explanations. It approximates the model's behavior in a small neighborhood around a specific data point by creating a local surrogate model.

A type of post-hoc explanation method that uses visualizations to highlight important parts of an input that contribute to a model's prediction. These maps typically show which areas of an image or text are most influential in determining the model's output.

Interactive visualizations that allow users to explore and understand complex models in a more engaging and interactive way. They enable users to modify inputs, observe how the model's predictions change, and gain insights into the decision-making process.