Intelligent Algorithms Lecture 2 PDF

Intelligent Algorithms AI401 Lec. 2 Prof. Tarek Moustafa Mahmoud Faculty of Computers & AI University of Sadat City  Artificial Neural Networks (ANNs) As discussed previously, ANN is an information processing technology inspired by human brain. ANN imi...

Intelligent Algorithms AI401 Lec. 2 Prof. Tarek Moustafa Mahmoud Faculty of Computers & AI University of Sadat City  Artificial Neural Networks (ANNs) As discussed previously, ANN is an information processing technology inspired by human brain. ANN imitates biological neural network processes. Neurons form various networks, and these networks have learning capacity, memorize and reveal the relationship between data. 2 Intelligent Algorithms – Prof. Tarek M Mahmoud  Elements of a Neural Network Input Layer: This layer accepts input features. It provides information from the outside world to the network, no computation is performed at this layer, nodes here just pass on the information(features) to the hidden layer. Hidden Layer: Nodes of this layer are not exposed to the outer world, they are part of the abstraction provided by any neural network. 3 Intelligent Algorithms – Prof. Tarek M Mahmoud  The hidden layer performs all sorts of computation on the features entered through the input layer and transfers the result to the output layer. Output Layer: This layer bring up the information learned by the network to the outer world. Activation Functions An activation function in a neural network defines how the weighted sum of the input is transformed into an output from a node or nodes in a layer of the network. In Artificial Neural Network, some different activation functions are used such as sigmoid, tanh, relu, softmax etc. 4 Intelligent Algorithms – Prof. Tarek M Mahmoud Sigmoid Function It is a logistic function, and this function is frequently used in deep learning. Value ranges are between 0 and 1. For binary classification problems, if values are between 0 and 0.5, label of my data is 0, otherwise 1. 5 Intelligent Algorithms – Prof. Tarek M Mahmoud Tanh Function Tanh (Hyperbolic Tangent Function) is trigonometric function and it returns between -1 and 1 values. 6 Intelligent Algorithms – Prof. Tarek M Mahmoud ReLU Function ReLU (Rectified Linear Activation Function) is a linear functions and that gives a value between 0 and infinite. 7 Intelligent Algorithms – Prof. Tarek M Mahmoud  Note that, weights and bias are learnable parameters in machine learning. These parameters are updated, and appropriate values are found. 8 Intelligent Algorithms – Prof. Tarek M Mahmoud  An activation function decides whether a neuron should be activated or not. This means that it will decide whether the neuron’s input to the network is important or not in the process of prediction using simpler mathematical operations. 9 Intelligent Algorithms – Prof. Tarek M Mahmoud  10 Intelligent Algorithms – Prof. Tarek M Mahmoud Note that, In regression problems we need one neuron as output layer In binary classification problems we need two neurons as output layer. In multi-class classification problems, the number of neurons in the output layer is equal to the unique classes, each representing 0/1 output for one class. 11 Intelligent Algorithms – Prof. Tarek M Mahmoud An example of single-input neuron As can been seen from this Figure, the neuron output can be calculated as: a =𝑓(𝑤𝑝+𝑏) Let 𝑤 = 3 , 𝑝 = 2 and 𝑏 = –1.5, what is the single-input neuron output ? a =𝑓(3∗2 + (−1.5) =𝑓(4.5) The actual output depends on the particular transfer function that is chosen. 12 Intelligent Algorithms – Prof. Tarek M Mahmoud Multiple Input Neuron The neuron has a bias, which is summed with the weighted inputs to form the net input 𝑛 : 13 Intelligent Algorithms – Prof. Tarek M Mahmoud This expression can be written in matrix form: where the matrix 𝐖 for the single neuron case has only one row. Now the neuron output can be written as: 14 Intelligent Algorithms – Prof. Tarek M Mahmoud Single Layer of 𝑺 Neurons 15 Intelligent Algorithms – Prof. Tarek M Mahmoud Three-Layer Network 16 Intelligent Algorithms – Prof. Tarek M Mahmoud The importance of neural networks Neural networks are one of the most popular machine learning models out there. They are so useful that the field has its own name: deep learning. Deep learning has numerous applications in the most cutting- edge areas of machine learning, including image recognition, Natural Language Processing, medicine, and self-driving cars. Generally, neural networks are useful for classification and regression. 17 Intelligent Algorithms – Prof. Tarek M Mahmoud Regression: The target output is a real number or a whole vector of real numbers. The price of a stock in 6 months time. The temperature at noon tomorrow. Classification: The target output is a class label. The simplest case is a choice between 1 and 0. We can also have multiple alternative labels. 18 Intelligent Algorithms – Prof. Tarek M Mahmoud Examples Recognizing patterns: Objects in real scenes Facial identities or facial expressions Spoken words Recognizing anomalies: Unusual sequences of credit card transactions Unusual patterns of sensor readings in a nuclear power plant Prediction: Future stock prices or currency exchange rates Which movies will a person like? 19 Intelligent Algorithms – Prof. Tarek M Mahmoud  Artificial Neural Network - Building Blocks Processing of ANN depends upon the following three building blocks: ❑ Network Topology ❑ Adjustments of Weights or Learning ❑ Activation Functions 20 Intelligent Algorithms – Prof. Tarek M Mahmoud Network Topology A network topology is the arrangement of a network along with its nodes and connecting lines. According to the topology, ANN can be classified to the following: ❑ Feedforward artificial neural networks. ❑ Perceptron and Multilayer Perceptron neural networks. ❑ Radial basis functions artificial neural networks. 21 Intelligent Algorithms – Prof. Tarek M Mahmoud ❑ Recurrent neural networks. ❑ Modular neural networks. 22 Intelligent Algorithms – Prof. Tarek M Mahmoud