w04artificialneuralnetworks_312b9688cb3401ada168c29dfa7f72e4_.pdf

Artificial Neural Networks Deep Learning Dr. Mohammed SALEM Dr. Mohammed SALEM 1 Biological & Artificial Neurons Dr. Mohammed SALEM 2 What are neurons ? What are neurons ? A neuron can be defined as the basic computational unit of the human brain. Neurons are the fundamental units of our brain and n...

Artificial Neural Networks Deep Learning Dr. Mohammed SALEM Dr. Mohammed SALEM 1 Biological & Artificial Neurons Dr. Mohammed SALEM 2 What are neurons ? What are neurons ? A neuron can be defined as the basic computational unit of the human brain. Neurons are the fundamental units of our brain and nervous system. Neurons are information messengers. They use electrical impulses and chemical signals to transmit information between different areas of the brain, and between the brain and the rest of the nervous system. Our brain encompasses approximately 100 billion neurons. Dr. Mohammed SALEM 3 What are neurons ? Each and every neuron is connected to one another through a structure called a synapse. A synapse is accountable for receiving input from the external environment, sensory organs for sending motor instructions to our muscles, and for performing other activities. Dr. Mohammed SALEM 4 How neurons in our brain actually work? A neuron can also receive inputs from the other neurons through a branchlike structure called a dendrite. These inputs are strengthened or weakened; that is, they are weighted according to their importance and then they are summed together in the cell body called the soma. From the cell body, these summed inputs are processed and move through the axons and are sent to the other neurons. Dr. Mohammed SALEM 5 How the artificial neurons work? Let's suppose we have three inputs x1, x2, x3 to predict output y. These inputs are multiplied by weights w1, w2, w3 and are summed together as follows: x1 w1 + x2 w2 + x3 w3 Dr. Mohammed SALEM 6 How the artificial neurons work? All of the inputs are not equally important in calculating the output y. Let's say that x1 is more important in calculating the output compared to the other two inputs. Then, we assign a higher value to w1 than the other two weights. Weights are used for strengthening the inputs Dr. Mohammed SALEM 7 How the artificial neurons work? After multiplying inputs with the weights, we sum them together and we add a value called bias, b: z = (x1 w1 + x2 w2 + x3 w3) + b We know that the equation of a straight line is given as: z = mx + b Here m is the weights (coefficients), x is the input, and b is the bias (intercept). Dr. Mohammed SALEM 8 What is the difference between neurons and linear regression? In neurons, we introduce non-linearity to the result, z, by applying a function f (. ) called the activation or transfer function. Thus, our output becomes: y = f (z) The neuron takes the input, x, multiples it by weights, w, and adds bias, b, forms z, and then we apply the activation function on z and get the output, y. Dr. Mohammed SALEM 9 How does the neuron activation work? Dr. Mohammed SALEM 10 ANN Layers Artificial neurons are arranged in layers. Each and every layer will be connected in such a way that information is passed from one layer to another. Neurons in the same layer do not have any connections. We use the term nodes or units to represent the neurons in the ANN. Dr. Mohammed SALEM 11 ANN Layers A typical ANN consists of the following layers: Input layer Hidden layer Output layer Dr. Mohammed SALEM 12 Input Layer The input layer is where we feed input to the network. The number of neurons in the input layer is the number of inputs we feed to the network. Each input will have some influence on predicting the output. However, no computation is performed in the input layer; it is just used for passing information from the outside world to the network. Dr. Mohammed SALEM 13 Hidden Layer Any layer between the input layer and the output layer is called a hidden layer. It processes the input received from the input layer. The hidden layer is responsible for deriving complex relationships between input and output. The hidden layer identifies the pattern in the dataset. It is majorly responsible for learning the data representation and for extracting the features. Dr. Mohammed SALEM 14 Hidden Layer There can be any number of hidden layers; however, we have to choose a number of hidden layers according to our use case. For a very simple problem, we can just use one hidden layer, but while performing complex tasks such as image recognition, we use many hidden layers, where each layer is responsible for extracting important features. The network is called a deep neural network when we have many hidden layers. Dr. Mohammed SALEM 15 Output Layer The hidden layer sends its result to the output layer. The output layer emits the output. The number of neurons in the output layer is based on the number and type of problem to be solved. Dr. Mohammed SALEM 16 Output Layer If it is a binary classification, then the number of neurons in the output layer is one that tells us which class the input belongs to. If it is a multi-class classification say, with five classes, then the number of neurons in the output layer is five, each emitting the probability. If it is a regression problem, then we have one neuron in the output layer. Dr. Mohammed SALEM 17 Exploring activation functions It is used to introduce non-linearity in neural networks. The aim of the activation function is to introduce a nonlinear transformation to learn the complex underlying patterns in the data. y = f (z) = f (input × weights + bias) Dr. Mohammed SALEM 18 The Sigmoid (Logistic) Function The sigmoid function scales the input value between 0 and 1. The sigmoid function can be defined as follows: import numpy as np def sigmoid(x): return 1/ (1+np.exp(-x)) Dr. Mohammed SALEM 19 The Tanh Function A hyperbolic tangent (tanh) function outputs the value between -1 to +1. It is a differentiable and monotonic. def tanh(x): numerator = 1-np.exp(-2*x) denominator = 1+np.exp(-2*x) return numerator/denominator Dr. Mohammed SALEM 20 The Rectified Linear Unit (ReLU) function The ReLU function is another one of the most commonly used activation functions. It outputs a value from 0 to infinity. It returns zero when the value of x is less than zero and returns x when the value of x is greater than or equal to zero. Dr. Mohammed SALEM 21 The Rectified Linear Unit (ReLU) function The snag for being zero for all negative values is a problem called dying ReLU, and a neuron is said to be dead if it always outputs zero. def ReLU(x): if x