Artificial Intelligence Overview

Artificial Intelligence

• Artificial intelligence encompasses a broad field.
• Machine learning is a subset of artificial intelligence.
• Deep learning is a subset of machine learning.

Model of an Artificial Neuron

• A diagram shows the components of an artificial neuron.
• Input values (x₁, x₂, ..., xₙ) are multiplied by weights (w₁, w₂, ..., wₙ).
• The weighted inputs are summed to produce a net input.
• An activation function (f) processes the net input, producing the output (y).

Multi-Layer Net

• A diagram shows a multi-layer neural network.
• Input layer processes input data.
• Hidden layers perform computations.
• Output layer produces output.
• The network structure allows complex computations.

Supervised Learning

• Supervised learning problems include classification and regression.
• Classification problems use categorical output variables (e.g., "red", "blue", "disease" or "no disease").
• Regression problems use continuous output variables (e.g., "dollars," "weight").

Common Supervised Machine Learning Algorithms

• Decision Trees
• K Nearest Neighbors
• Linear SVC (Support Vector Classifier)
• Logistic Regression
• Linear Regression

Classification Model Steps using Scikit-Learn

• Import libraries from scikit-learn
• Load the Iris dataset
• Split into training and testing sets
• Instantiate an Support Vector Classifier (SVC) with a linear kernel;
• Train the classifier
• Make predictions
• Evaluate the classifier

Evaluating Classification Methods

• Predictive Accuracy: The number of correct classifications divided by the total number of test cases..
• Efficiency: Time to construct the model and use the model.
• Robustness: Handling noise and missing values.
• Scalability: Efficiency in databases.
• Interpretability: Understandable insights provided by the model (e.g., number of rules, size of the tree.)

Classification Model

• The model encompasses steps for performing classification.
• Different stages include; library importation, data preparation, model definition, model compilation, training, evaluation, and plotting.

Machine vs. Deep Code

• Shows different code structures for machine learning and deep learning, indicating the importation of libraries, the loading of datasets, and the creation of classifiers.

Activation function Softmax Layer

• The softmax layer applies softmax activations, producing probability values between 0 and 1.
• Softmax values are the logits.
• The summation of all probabilities equals to 1.

Activation Functions

• Non-linear activations are needed for complex data representations in Neural Networks.
• Neural Networks (NNs) with more layers and neurons can approximate complex functions.
• More neurons improve data representation; however, it may cause overfitting.

Activation: Sigmoid Function

• The sigmoid function maps any real value to a range between 0 and 1, interpreted as firing rate.
• It is common in Neural Networks for its characteristics.
• But the gradients become almost zero when the input gets very small or very large.

Activation: Tanh Function

• The tanh (hyperbolic tangent) function maps real values to the range between -1 and +1.
• It is zero-centered, which is preferred to the sigmoid function.
• Tanh also has characteristics, including vanishing gradients when the input gets very small or very large.

Activation: ReLU (Rectified Linear Unit)

• The ReLU activation function thresholds the input at zero (maps all negative values to 0 and positive inputs to themselves), suitable for Modern Deep Neural Networks because of its efficiency.
• It speeds up computations.
• Compared to other functions, has reduced overfitting and a reduced training time.

Activation: Leaky ReLU

• Leaky ReLU is a variation of ReLU.
• Instead of outputting 0 when the input is negative, it has a small negative slope (e.g., 0.01).
• This modification resolves the "dying ReLU" problem.

Activation: Linear Function

• The linear function outputs a signal that is proportional to the input.
• If the constant is 1, it's an "identity function."
• Common in regression problems, when the output needs to be a real number.

Training Neural Networks

• Train NNs involve setting the parameters with a gradient descent algorithm (GD), making the predictions closest to the true values.
• Gradient descent updates the parameters by moving in the opposite direction of the gradient..

Data Preprocessing

• Data preprocessing (e.g., mean subtraction, normalization) helps convergence.
• Data normalization/zero-centering helps convergence in training neural networks.
• Normalization may use standard deviation or scale to the range of 0 to 1.

Batch Normalization

• Batch normalization speeds up training (convergence).
• Acts similar to data preprocessing by calculating the mean and variance of a batch of input data and normalizing.
• Useful during training neural networks to alleviate initialization issues and make it easier for the algorithm to learn.

Loss Functions

• Loss functions (e.g., cross-entropy for classification, mean squared error for regression) measure the difference between predicted and actual values, used in neural networks to drive optimization.
• Classification uses a sigmoid activation function on the output layer to return probabilities of a categorical label.
• Regression uses a linear activation function (or sigmoid, when appropriate) to directly compute the output, which may be a real number or a continuous range of numbers.

Training NNs

• An algorithm to optimize a loss function to train neural networks.

Gradient Descent

• Gradient descent helps find the minimum of a loss function (and optimal parameters).
• Steps include random initialization, calculating the gradient, and updating parameters using a learning rate.