Introduction to Machine Learning Concepts

The ability of a computer to learn without explicit programming, using data to make predictions or solve problems.

Focus on finding relationships and explaining them, emphasizing interpretability through parameters.

Focus on making predictions and generalizing patterns from data, often with a large number of parameters.

Used to predict a numerical value (e.g., house price) based on input features.

A type of ML where the model learns from labeled data with correct answers (X, Y) to predict similar outcomes in new data.

A type of ML where the model explores unlabeled data (X) to discover patterns and structures.

A type of ML where the model learns through trial and error, receiving rewards or penalties for its decisions.

ML provides powerful tools for pattern recognition and prediction, but limitations include potential lack of causality and suitability for specific problems.

Machine learning models used for both classification and regression tasks, particularly important in deep learning.

The input layer represents the features you want to use to predict something. Hidden layers process information, and the output layer gives you the result.

The number of hidden layers determines the network's complexity. A shallow network has one hidden layer, while a deep network has two or more.

Weights determine how information flows between nodes, while bias nodes add an intercept.

The loss function measures the error between predictions and actual values. It helps guide the network to improve accuracy.

Backpropagation is an iterative process of adjusting weights to lower the loss function. It uses gradient descent methods like Adam and RMSprop.

Hyperparameter tuning is like fine-tuning a machine to get optimal performance. Key hyperparameters include the number of layers and nodes, batch size, learning rate, and regularization.

K-Fold cross validation provides a more robust evaluation of generalization performance than a simple train/test split. Data is divided into K folds, and the model is trained and tested on different combinations of folds.

A technique where multiple decision trees are sequentially trained, with each tree focusing on correcting errors from the previous ones.

A representation of discrete data (like words or images) as continuous vectors in a lower-dimensional space.

The relationship where a change in one variable (X) directly causes a change in another variable (Y), while other factors remain constant.

A process where you input data into a model and it predicts an output based on the learned patterns, even for data points it hasn't seen before.

The process of taking data that has missing or inaccurate information and filling in those gaps using statistical methods.

A measure of how accurately a model predicts the target variable, taking into account the complexity of the model.

A machine learning technique where multiple models are combined to make more accurate predictions.

An ensemble model combines multiple "weak" models to create a stronger model. It's based on the idea that diversity reduces bias and increases generalization, similar to the "wisdom of the crowd".

A type of ensemble model that utilizes decision trees to solve classification and regression problems. Random Forests combat the sensitivity of decision trees to overfitting by adding diversity through bagging and random patching.

A method for building Random Forests that involves generating multiple bootstrap datasets by randomly sampling with replacement. Each bootstrap dataset is used to build a decision tree.

A method for building Random Forests that involves using randomly selected features to split nodes in decision trees. This further increases diversity and reduces overfitting.

A type of ensemble model that sequentially trains multiple models, where each model focuses on correcting the errors made by the previous model. This iterative process leads to a stronger model.

The number of trees in a Random Forest model. Increasing the number of trees generally improves performance up to a certain point.

The maximum number of features used for splitting nodes in a decision tree within a Random Forest model. It controls the amount of information used for each split.

A technique for improving ML model performance and generalization. It involves splitting the data into multiple folds and training and evaluating the model on different combinations of folds.

A method that uses game theory to determine the contribution of each input feature to a model's output. It fairly distributes the output among features based on their marginal contributions in different combinations.

A property of Shapley Values ensuring that the sum of all feature contributions equals the model's final prediction. It ensures all contributions are accounted for.

A property of Shapley Values where two features with identical contributions receive the same score, regardless of their order or combination with other features.

A property of Shapley Values ensuring that if a feature contributes more to a prediction, its Shapley Value doesn't decrease even if other features are added or removed.

A visual tool presenting the contributions of each feature for a single prediction. It shows how features cumulatively build up to the final prediction.

A visual tool showing Shapley Values for multiple data points. It helps understand the distribution and significance of each feature across various instances.

A method that uses local explanations for individual predictions by perturbing features and observing the resulting change in predictions. It's fast but can be inaccurate for complex models.

A method that provides global explanations by calculating Shapley Values to determine feature contributions. It's more accurate but computationally intensive.

AI systems should be developed and used in ways that are ethical, fair, and responsible. This area focuses on addressing potential harms like discrimination, privacy violations, and misinformation.

Privacy risks arise from AI systems that handle sensitive data, potentially leading to misuse, unauthorized access, or identity leakage. Example: Strava heatmaps revealing sensitive user movements.

Large language models (LLMs) pose privacy challenges because they learn from massive datasets. Sensitive data leaks and misuse are potential risks.

Explainable AI (XAI) aims to provide clear and understandable explanations for how AI systems reach their decisions. This helps build trust and ensure responsible use.

XAI is crucial for climate modeling as it helps understand complex climate simulations and interpret predictions, supporting informed decision-making.

AI can support energy efficiency in buildings by optimizing energy consumption, analyzing building designs, and suggesting improvements to reduce energy waste.

AI can be used to simulate the impacts of policies on greenhouse gas emissions, helping policymakers understand potential outcomes and evaluate different approaches.

AI applications can influence policy choices towards a more sustainable future by providing data-driven insights and supporting better decision-making.