Week 13 COE305 Machine Learning

Week 13 COE305 Machine Learning

• Course covers Artificial Neural Networks (ANN)
• ANNs mimic human behavior through neuron activation
• ANNs use artificial neurons that interconnect
• Perceptron is a fundamental ANN concept
• Perceptrons allow multiple inputs to create a single output (e.g., loan approval)
• ANNs were theorized in the 1950s
• ANN structure is based on the human brain's biological neural network
• ANNs don't use activation functions, only nodes
• ANNs have inputs, nodes, weights, output

ML vs ANN

• Machine Learning (ML): A subset of AI focusing on algorithms learning from data. It includes various methods like supervised, unsupervised, and reinforcement learning, encompassing simpler algorithms like regression and decision trees.

• Artificial Neural Networks (ANN): A subset of ML inspired by biological neural networks. ANNs focus on neural network-based approaches, often more complex with multiple layers and parameters.

• Data Requirements: ML performs well with smaller to medium-sized datasets, while ANNs require significantly larger datasets to effectively learn.

• Interpretability: ML models are generally easier to interpret compared to ANNs, which can be more challenging to understand due to their complexity.

• Applications: ML finds use in predictive analytics, fraud detection, recommendation systems, and basic classifications. ANNs are crucial for image recognition, speech recognition, and autonomous vehicles.

• Training Time: Simple ML models typically train faster compared to intricate ANN models.

• Key Algorithms: Linear regression, logistic regression, decision trees are examples of common ML algorithms. Feedforward NN, Convolutional NN, and Recurrent NN. are examples of ANN.

• Computational Power: ML benefits from lower computational power for simple models. Compared to ANNs, which often require substantial computational resources, particularly for complex deep learning applications.

Basic Architecture of Neural Networks

• Neural networks consist of interconnected nodes, which can be represented with numbers, like salary, tenure etc.
• The network's structure includes input nodes, hidden layers, and output nodes. They help with complex processing.
• Input layer receives information.
• Hidden layers process the received data using summation and activation functions; they connect to other layers and nodes. The hidden layer's main function is to process data received from the inputs, using a function-activation, and transmit it to the output layer to complete the task using other layers.
• The output layer outputs the results; it's the final stage.
• Connections between nodes are weighted to modulate signals.
• Input values are multiplied with weights, then combined, and passed through an activation function to get the network's output (e.g. affordability example).

Components of Neural Networks

• Layers: Input, hidden layers, output, multiple layers help complex processing
• Weights: Adjust the importance of inputs in determining the output.
• Bias: Shifts the activation function to fit the provided data; it adjusts the decision boundary to improve the model's accuracy.
• Activation Functions: Functions applied to the weighted sum of inputs, defining the output values, crucial for non-linear operations. Popular activation functions include sigmoid, ReLU, tanh.

Activation Functions

• Threshold/Binary Step: Output is 1 if input > 0, else 0.
• Sigmoid/Logistic: Transforms values between 0 and 1, useful for classification.
• ReLU (Rectified Linear Unit): Output = input if > 0, else 0.
• Tanh (Hyperbolic Tangent): Output values between -1 and 1.
• Leaky ReLU: A variant of ReLU that allows a small but non-zero output for negative inputs.

Feed-forward Neural Network (FFNN)

• FFNNs have a direct path from input to output without loops.
• Input values are weighted and summed.
• The activation function processes the sum, determining the output.
• Errors are calculated, and weights are modified via backpropagation to decrease error.

Neural Network with Backpropagation

• Backpropagation adjusts weights to minimize errors in the model's predictions by learning from previous errors to improve predictions in the future.
• The "learning rate" determines the step size during weight adjustments to reach minima.
• Used after the activation function completes; it's an iterative process.

Total Error Calculation

• The total error measures the differences between predicted and actual values across all instances in the dataset by adding up errors for each instance, and it's a way to quantify the inaccuracy of the models, indicating the model's capability.
• Calculation is based on an error function, such as log loss in binary classification problems.