Artificial Intelligence Overview

Study Notes

Supervised learning problems include classification and regression.
Classification problems use categorical output variables (e.g., "red", "blue", "disease" or "no disease").
Regression problems use continuous output variables (e.g., "dollars," "weight").

Predictive Accuracy: The number of correct classifications divided by the total number of test cases..
Efficiency: Time to construct the model and use the model.
Robustness: Handling noise and missing values.
Scalability: Efficiency in databases.
Interpretability: Understandable insights provided by the model (e.g., number of rules, size of the tree.)

The model encompasses steps for performing classification.
Different stages include; library importation, data preparation, model definition, model compilation, training, evaluation, and plotting.

Shows different code structures for machine learning and deep learning, indicating the importation of libraries, the loading of datasets, and the creation of classifiers.

The softmax layer applies softmax activations, producing probability values between 0 and 1.
Softmax values are the logits.
The summation of all probabilities equals to 1.

Non-linear activations are needed for complex data representations in Neural Networks.
Neural Networks (NNs) with more layers and neurons can approximate complex functions.
More neurons improve data representation; however, it may cause overfitting.

The sigmoid function maps any real value to a range between 0 and 1, interpreted as firing rate.
It is common in Neural Networks for its characteristics.
But the gradients become almost zero when the input gets very small or very large.

The tanh (hyperbolic tangent) function maps real values to the range between -1 and +1.
It is zero-centered, which is preferred to the sigmoid function.
Tanh also has characteristics, including vanishing gradients when the input gets very small or very large.

The ReLU activation function thresholds the input at zero (maps all negative values to 0 and positive inputs to themselves), suitable for Modern Deep Neural Networks because of its efficiency.
It speeds up computations.
Compared to other functions, has reduced overfitting and a reduced training time.

Leaky ReLU is a variation of ReLU.
Instead of outputting 0 when the input is negative, it has a small negative slope (e.g., 0.01).
This modification resolves the "dying ReLU" problem.

Train NNs involve setting the parameters with a gradient descent algorithm (GD), making the predictions closest to the true values.
Gradient descent updates the parameters by moving in the opposite direction of the gradient..

Data preprocessing (e.g., mean subtraction, normalization) helps convergence.
Data normalization/zero-centering helps convergence in training neural networks.
Normalization may use standard deviation or scale to the range of 0 to 1.

Batch normalization speeds up training (convergence).
Acts similar to data preprocessing by calculating the mean and variance of a batch of input data and normalizing.
Useful during training neural networks to alleviate initialization issues and make it easier for the algorithm to learn.

Loss functions (e.g., cross-entropy for classification, mean squared error for regression) measure the difference between predicted and actual values, used in neural networks to drive optimization.
Classification uses a sigmoid activation function on the output layer to return probabilities of a categorical label.
Regression uses a linear activation function (or sigmoid, when appropriate) to directly compute the output, which may be a real number or a continuous range of numbers.

Gradient descent helps find the minimum of a loss function (and optimal parameters).
Steps include random initialization, calculating the gradient, and updating parameters using a learning rate.