Smart Systems and Computational Intelligence Lecture 9 PDF

# Smart Systems and Computational Intelligence ## Lecture 9 Smart Systems and Computational Intelligence Course ## Neuro-Fuzzy Systems ### Introduction - Neuro-fuzzy systems combine the properties and benefits of NNs and FL. - They are soft computing methods that combine neural networks and fuzz...

# Smart Systems and Computational Intelligence ## Lecture 9 Smart Systems and Computational Intelligence Course ## Neuro-Fuzzy Systems ### Introduction - Neuro-fuzzy systems combine the properties and benefits of NNs and FL. - They are soft computing methods that combine neural networks and fuzzy concepts in various ways - ANNs are used in part for low level perceptive and signal integration - The fuzzy part represents higher level reasoning aspects - Fuzzy systems transfer the vague fuzzy form of human reasoning to mathematical systems - The use of IF-THEN rules in fuzzy systems gives us the possibility of easily understanding the information modeled by the system - Most of the knowledge in fuzzy systems is obtained from human experts, which is a disadvantage, because not every human expert can or wants to share their knowledge. - ANNs can learn from experience but most of the topologies do not allow us to clearly understand the information learned by the networks. - ANNs are incorporated into fuzzy systems to form neuro-fuzzy systems, which can acquire knowledge automatically by learning algorithms of NNs. - Neuro-fuzzy systems have the advantage over fuzzy systems that the acquired knowledge is more meaningful to humans. - Clustering is another technique for neuro-fuzzy systems, which is used to initialize unknown parameters such as the number of fuzzy rules or the number of membership functions for the premise part of the rules. They are also used to create dynamic systems and update the parameters of the system. - The operation of the neuro-fuzzy system is expressed as linguistic fuzzy expressions and learning schemes of NNs are used to learn the system. - Neuro-fuzzy systems allow incorporation of both numerical and linguistic data into the system. - The neuro-fuzzy system is also capable of extracting fuzzy knowledge from numerical data - The neuro-fuzzy systems can be divided into two main groups: - Neural fuzzy inference systems - Fuzzy neural networks ### 1. Neural Fuzzy Inference Systems - The origin of neural fuzzy inference systems is to incorporate neural concepts, such as learning and parallelism, into fuzzy logic inference systems. - The fuzzy inference can be implemented in two ways - One network can be used that realizes the whole fuzzy inference - Each fuzzy rule is realized using a neural network, when the fuzzy inference is the result of several neural networks. ### 2. Fuzzy Neural Networks - Fuzzy ideas are incorporated into NNs - The weighted sum of the neuron is replaced by a corresponding fuzzy operation - The operation of the neuro-fuzzy system was the same as the NN - Fuzzy neural networks consist of two components in the same system - A fuzzy system - A neural network. - The fuzzy system can be either a fuzzy inference block that converts linguistic information for the NN or the NN can drive the fuzzy inference block. - A common approach to neuro-fuzzy systems is to fuzzify the learning algorithms of different NN paradigms - For example, the learning rate coefficient of Kohonen's self-organizing map is treated as a fuzzy membership value of the current input sample in the class of each neuron. ### How ANNs and FSs can interact? There are two main ways that ANNs and fuzzy systems can interact: 1. The fuzzification of NNs; weight level, TF level, learning algorithm level. 2. Giving fuzzy systems features of NNs; the ANN is used to learn membership functions or rules for a given fuzzy system. ### Classes of Neuro-Fuzzy Systems: 1. Co-operative: Neural algorithms learn membership functions or rules or both. 2. Concurrent: the two techniques are applied after one another as pre- or post-processing. 3. Hybrid: the fuzzy system is represented as an NN to take advantage of learning algorithms inherited from ANNs. ## Fuzzy Neural Networks - Integrating NNs and fuzzy systems into a single new soft computing model gives hopes of exploiting their complementary nature by reinforcing the good points and avoiding their respective shortcomings. ### Fuzzy Neurons: - Fuzzy models of ANNs can be constructed by using fuzzy operations at the single neuron level. - The fuzzy operations used are the union and intersection of fuzzy sets, and more generally, t-norms and t-conorms. - A variety of fuzzy neurons can be obtained by applying fuzzy operations to connection weights, aggregation functions, or both of them. - A step toward the fuzzification of an ANN can be done by considering other forms A of the aggregation function according to the more general equation: - $y = g (A(w; x))$ - where g is the transfer function and y is the output signal of the neuron. ### Generalized Union/Intersection - Generalized Union - **t-norm** - Or called triangular norm. - Generalized Intersection - **t-conorm** - Or called s-norm. ### t-norm - Definition 1.2.1. A function $T : [0,1]^2 → [0,1]$ is called triangular norm (t-norm) if and only if it fulfills the following properties for all $ x, y, z ∈ [0,1]$. - (T1) $T(x, y) = T(y, x)$, the t-norm is commutative. - (T2) $T(T(x, y), z) = T(x, T(y, z))$, the t-norm is associative. - (T3) $x ≤ y ⇒ T(x,z)≤ T(y, z)$, the t-norm is monotone. - (T4) $T(x,1) = x$, a neutral element exists, which is 1. ### t-conorm - Definition 1.2.3. A function $S: [0,1]^2 → [0,1]$ is called triangular conorm (t-conorm) if and only if it fulfills the following properties for all x,y,z \in [0,1] - (S1) $S(x, y) = S(y, x)$, the t-conorm is commutative. - (S2) $S(S(x, y), z) = S(x,S(y, z))$, the t-conorm is associative. - (S3) $x ≤ y ⇒ S(x,z)≤ S(y, z)$, the t-conorm is monotone. - (S4) $S(x,0) = x$, a neutral element exists, which is 0. ### OR Fuzzy Neuron - $y = OR(x_1 AND w_1, x_2 AND w_2,..., x_n AND w_n)$ ### AND Fuzzy Neuron - $y = AND(x_1 OR w_1, x_2 OR w_2,..., x_n OR w_n)$ ### The OR/AND Neuron: - It is a combination of the AND and OR neurons into a two-layer structure. - The behavior of the net can be modulated by suitably weighting the output signals from the OR or the AND parts through setting or learning the connection weights c1 and c2. - The limiting cases are; - c1 = 0 and c2 = 1: the system reduces itself to a pure AND neuron. - c1 = 1, and c2 = 0:, the system reduces itself to a pure OR neuron. ### Multilayered Fuzzy Neural Networks: - Fuzzy neurons can be assembled together into multilayered networks. - Since the neurons are in general different, the construction gives rise to non-homogeneous neural networks. - For example; the first NN is composed of a hidden layer with p neurons (AND type) and an output layer with a single OR neuron. The input is constituted by 2n values. - The second possibility is to have OR neurons in the hidden layer and a single AND neuron in the output layer. ### Network Architecture: - The fuzzy logic inference system can be implemented as a five-layer NN. - Consider a system which has 2 inputs X1 and X2 and only one output y. - Rule 1: IF x1 is A1 AND x2 is B1, THEN y is z1, - Rule 2: IF x1 is A2 AND x2 is B2, THEN y is z2. ### Neuro-fuzzy classifier architecture: - The architecture of the neuro-fuzzy classifier is slightly different from the architecture used in function approximator; - Layer 1&2: have the identical function with the approximator. - Layer 3: Combination of firing strengths - Layer 4: Fuzzy outputs - Layer 5: Defuzzification Two first layers - Consider a neuro-fuzzy system using following fuzzy rules, - Rule 1: IF x1 is A1 AND x2 is B1, THEN CLASS is 1, - Rule 2: IF x2 is A2 AND x2 is B2, THEN CLASS is 2, - Rule 3: IF x1 is A1 AND x2 is B2, THEN CLASS is 1. ### Adaptive Neuro Fuzzy Inference System (ANFIS): - ANFIS implements a first order Sugeno-style fuzzy system. - It is a method for tuning an existing rule with a learning algorithm based on a collection of training data. - ANFIS constructs a fuzzy inference system whose membership function parameters are tuned using either a back propagation algorithm alone, or in combination with least squares method. - In this NN, the neuro fuzzy controller has 2 inputs (X & Y) and it has an o/p (Y). For each input 20 membership functions and also 20 rules in the rules base is considered, such as; - If X is Al and Y is B1 then fl=p1 X+q1Y+rl. - If X is A2 and Y is B2 then f2=p2 X+q2 Y+r2. ### An Example: NEFPROX - NEuro Fuzzy function apPROXimator - Three-layer feedforward network (no cycles in the network and no connections exist between layer n and layer n+j, with j>1 - input variables / hidden layer - fuzzy rules/output variables - Hidden and output units use t-norms and t-conorms as aggregation functions - The fuzzy sets are encoded as fuzzy connection weights and fuzzy inputs ### NEFPROX - The input units are labelled x₁..Xn, hidden rule units are called R₁...Rk and the output units are denoted as Y1 Ym - Each connection is weighted with a fuzzy set and is labelled with a linguistic term - Connection coming from the same input unit and having same label are weighted by the same common weight (shared weight). The same holds for the connections that lead to the same output unit - There is no pair of rules with identical antecedents ### NEFPROX – learning #### Structure Learning Algorithm 1. Select the next training pattern (s,t) from the training set. 2. For each input unit x; find the membership function μ(i)(s) such that - μ(i)(s) = max {μ(i)(s)}. je(1,....Pi) 3. If there is no rule $R$ with weights W(11, R) = μ(1) 1 ,..., W(xn, R) = μ(1) then create the node and connect it to all the output nodes. Hin 4. For each connection from the new rule node to the output nodes find a suitable fuzzy weight v(i) using the membership functions assigned to the output units y, such that vi (ti) = maxje{1,...} {v(i) (t;)} and v(i) (ty) ≥ 0.5. If the fuzzy set is not defined then create a new one v(i) (t₁) for the output variable y; and set W(R, Yi) = v(i) (t₁) new 5. If there are no more training patterns then stop rule creation; otherwise go to 1. 6. Evaluate the rule base and change the rule conclusions if appropriate. ### NEFPROX – learning #### Parameter Learning Algorithm 1. Select the next training pattern (s, t) from the training set and present it at the input layer. 2. Propagate the pattern forward through the hidden layer and let the output units determine the output vector o. 3. For each output unit y; determine the error dyi = ti - Oyi- 4. For each rule unit $R$ with output OR > O do: - Update the parameters of the fuzzy sets W (R, yi) using a learning rate parameter σ > 0. - Determine the change R = OR(1 - OR) Σycoutput layer (2W(R, y) (t₁) – 1). δ. - Update the parameters of the fuzzy sets W(x, R) using SR and o to calculate the variations. 5. If a pass through the training set has been completed and the convergence criterion is met then stop; otherwise go to step 1. ### Applications: Wheelchair Neuro-fuzzy Controller: - The position of the chair is manipulated so that it will avoid static and dynamic obstacles. The controller takes information from three ultrasonic sensors, and then decides the best direction that the wheelchair must follow. - The outputs of the neuro-fuzzy controller are the voltages sent to a system that generates a PWM to move the motors and the directions in which the wheel will turn. - The controller is based on trigonometric NNs and fuzzy cluster means. It follows a Takagi-Sugeno inference method, but instead of using polynomials on the defuzzification process it also uses trigonometric neural networks (T-ANNs). ### Wheelchair Neuro-fuzzy Controller: - Rules - If - Then ### The IF-THEN rules for the direction controller 1. IF Degree is Small & Direction is Left THEN PWMR IS Very Few, PWM₁ IS Very Few, DIRR is CCW, DIR₁ is CW. 2. IF Degree is Small & Direction is Center THEN PWMR IS Very Few, PWML IS Very Few, DIRR is NC, DIR₁ is NC. 3. IF Degree is Small & Direction is Right THEN PWMR IS Very Few, PWML IS Very Few, DIRR is CW, DIR₁ is CCW. 4. IF Degree is Medium & Direction is Left THEN PWMR IS Some, PWML IS Some, DIRR is CCW, DIRL is CW. 5. IF Degree is Medium & Direction is Center THEN PWMR IS Some, PWML IS Some, DIRR is NC, DIR₁ is NC. 6. IF Degree is Medium & Direction is Right THEN PWMR IS Some, PWML IS Some, DIRR is CW, DIRL is CCW. 7. IF Degree is Large & Direction is Left THEN PWMR IS Very Much, PWML IS Very Much, DIRR IS CCW, DIRL is CW. 8. IF Degree is Large & Direction is Center THEN PWMR IS Very Much, PWML IS Very Much, DIRR IS NC, DIRL is NC. 9. IF Degree is Large & Direction is Right THEN PWMR IS Very Much, PWML IS Very Much, DIRR IS CW, DIRL is CCW. ### DEVELOPMENT OF NEURO FUZZY CONTROLLER ALGORITHM FOR AIR CONDITIONING SYSTEM - The neurofuzzy controller for air conditioning system takes two inputs (each has 4 fuzzy sets) from temperature and humidity sensors and controls the compressor speed. #### Rule base of neuro-fuzzy controller | Rules | Temperature | Humidity | Compressor speed | |---|---|---|---| | 1. | Very Low | Dry | Off | | 2. | Very Low | Comfortable | Off | | 3. | Very Low | Humid | Off | | 4. | Very Low | Sticky | Low | | 5. | Low | Dry | Off | | 6. | Low | Comfortable | Off | | 7. | Low | Humid | Low | | 8. | Low | Sticky | Medium | | 9. | High | Dry | Low | | 10. | High | Comfortable | Medium | | 11. | High | Humid | Fast | | 12. | High | Sticky | Fast | | 13. | Very High | Dry | Medium | | 14. | Very High | Comfortable | Fast | | 15. | Very High | Humid | Fast | | 16. | Very High | Sticky | Fast | ### Any Questions??? ### Thanks