Nature-Inspired Optimization & Fuzzy Logic

Concepts Covered in Lecture 4

• Nature-inspired Optimization Tools are covered
• Discussion of the Optimization of Fuzzy Reasoning Tools
• Introduction to Optimization related to Fuzzy Clustering

Nature-inspired Optimization Tools

• Genetic Algorithms (GA)
• Genetic Programming (GP)
• Evolution Strategies (ES)
• Evolutionary Programming (EP)
• Particle Swarm Optimization (PSO)
• Ant Colony Optimization (ACO)
• Artificial Immune System (AIS)
• Artificial Bee Colony (ABC)
• Other optimization tools also exist

Genetic Algorithm (GA)

• The working cycle of the Genetic Algorithm involves initializing a population of solutions, assigning fitness to all solutions, reproduction, crossover, and mutation.
• The process iterates until the maximum generation count is reached.

Fuzzy Logic Controller (FLC)

• Fuzzy Logic Controller (FLC) performance depends on its knowledge base (KB).
• The knowledge base is comprised of the Data Base (DB) and Rule Base (RB).
• Tuning the KB of the FLC can be done through Nature-inspired Optimization tools based tuning in either an offline or online manner.

GA-based Tuning of Manually Constructed FLC

• A binary-coded Genetic Algorithm (GA) can be used to obtain an optimal DB and RB of a fuzzy reasoning tool.
• DB = Data Base
• RB = Rule Base
• Example case uses in inputs: l1 and l2
• Example case uses one output: O.
• Membership function distributions for inputs and output are graphically represented.
• Optimization is done with a set of training cases in a table.

GA-string Population

• An initial population of the Binary-Coded Genetic Algorithm (BCGA) is created at random.
• Five bits are assigned to represent each of b values (b1, b2, b3).
• Sixteen bits represent the Rule Base (RB) of the fuzzy reasoning tool.
• The variable "b" values vary through the ranges:
  • 2.0 ≤ b1 ≤ 4.0
  • 5.0 ≤ b2 ≤ 15.0
  • 0.5 ≤ b3 ≤ 1.5
• Deviation in prediction is determined for the first training case via the first GA-string.

Calculation to Determine Real b1 Value

• Example Real Value of $b_1$ can be found with the following steps
  • $b_1= 3.419355$
  • $b_2= 9.193548$
  • $b_3= 1.370968$
• Modified membership function distributions are determined

Rule Base (RB)

• The Rule Base is found to be as follows in the given example from the substring 1000101010111001.

Fuzzy Rules

• Firing present rules of example for input values are as follows
• $I_1$ = 10.0
• $I_2$ = 28.0.
• If $I_1$ is M and $I_2$ is LW then O is LW,
• If $I_1$ is H and $I_2$ is LW then O is M.
• Membership values are calculated

Membership Calculation

• $\mu_M= 0.83$
• $\mu_{LW} = 0.13$

$1^{st}$ Fired Rules

• This shows the membership calculation of $I_1$ compared to M and $I_2$ compared to LW
• Variable X = 1.19
• $Area A_1$= 0.1547
• $C_1$ = 2.595
• $Area A_2$ = 0.01176
• $C_2$ = 3.25
• The Final Total Area and Centre Point is then calulated with second fired rule

$2^{nd}$ Fired Rules

• This shows the membership calculation of $I_1$ compared to H and $I_2$ compared to LW

Application of Rules

• The following values are used to calculate the Crisp Output by using the centre of sums method for defuzzification
• Total area A" = 0.3068
• Centre or the fired area C" = 3.370968
• Calculation:
  • Crisp output O = 3.1138

Deviation Values

• Deviation is calculated, corresponding to the first training case: d1 = |3.5 – 3.1168| = 0.3832.
• All training cases are passed.
• Deviation values are determined.

String Calculation

• String values can be calculated by : $d = \frac{{\sum_{i=1}^{T} d_i }}{T}$

Automatic Design of FLC using GA

• Same numerical example that used in Approach 1 is used in Approach 2 with the RB missing.
• There are 4 x 4 = 16 possible combinations with four linguistic terms for each of the two variables.
• Sixteen rules are not pre-defined and this task of determining RB is given to the Genetic Algorithm.
• Two bits may be used to represent each of them with four linguistic terms used to represent the output.
  • LW = 00
  • M = 01
  • H = 10
  • VH = 11
• GA-string will be 63-bits long = 5 + 5 + 5 + 16 + 2 x 16.

GA String Example

• Based on previously covered info, the following applies
• $b_1, b_2$ and $b_3$ are found to be equal to 3.419355, 9.193548 and 1.370968
• Modified membership function distributions of the input and output variables will be the same with the ones shown in Fig. A
• With sub-strings, it indicates that the following rules are present in the RB of the FLC
• Rules represented by the GA-substring, Table B will be shown

Final Calculation

• If $I_1$ is M AND $I_2$ is LW Then O is M
• If $I_1$ is H AND $I_2$ is LW Then O is H
• It uses the Mamdani approach with used fuzzified output that corresponds to the input variables.
• The Center of Sums method of de-fuzzification to determine its crisp value.
• The controller output is found to be equal to 4.056452
• Deviation calculation gives the following output
• ||3.5 − 4.056452| = 0.556452

Deviation

• Values of absolute deviation can be determined for 2nd, 3rd,...T-th training scenarios, using the similar procedure.
• The average deviation in predictions d = (∑ἶ=1 di)/T

Fitness

• The fitness of the first GA-string, f1 is made equal to d.

GA population

• The population of GA-strings is modified using reproduction, crossover, and mutation operators.
• A GA-optimized Rule Base of the FLC may contain redundant rules.
• The concept of importance factor has been used in order to identify the redundant rules using worth and its probability of occurrence.

Optimization related to Fuzzy Clustering

• Maximize distinctness and compactness.
• Minimize the number of outliers.

Fuzzy C-Means Clustering

• Determines the number of clusters to be made.
• Determines the initial matrix of membership values.
• Determines the level of cluster fuzziness.

Implementation

• The said variables are encoded in the GA-string. Their optimal values can be obtained through a number of iterations.

Entropy-based Fuzzy Clustering

• Includes parameter α, indicating the relationship between Euclidean distance and similarity.
• Includes parameter β for the threshold value of similarity.
• Includes parameter γ representing Outliers.

Concepts Covered in Lecture 5

• Biological and Artificial Neurons
• Artificial Neural Networks
• Supervised and Un-supervised Learning
• Incremental and Batch modes of Training

Introduction to Neural Networks

• Proposed by McCulloh and Pitts in 1943.
• Biological nervous system is composed of interconnected processing units called neurons operating in parallel.
• The human brain contains approximately 10^11 neurons and is a highly complex parallel computer.

Biological Neuron Structure

• Consists of dendrites (a bush of thin fibers).
• Consists of a cell body.
• Consists of an axon.
• Consists of a synapse.

Artificial Neuron Structure

• Includes inputs ($I_j$).
• Includes Weights ($W_{nj}$).
• Includes Bias ($b_j$).
• Includes Summing Junctions.
• Includes an Activation/Transfer function (f)

Transfer Functions

• Hard-limit
  • O = [0.0, if u < 0.0 1.0, otherwise
• Linear
  • O = u
• Log-sigmoid
  • $\frac{1}{1 + e^{-au}}$
• Tan-sigmoid
  • O = $\frac{e^{au} - e^{-au}} {e^{au} + e^{-au}}$

One Layer of Neural Network

• Consists of a weight array
• Activation is determined via the formula $O_k= f(\sum_{j=1}^n I_jW_{jk} + b_k)$

Neural Networks

• Made of multiple layers of Neurons
• Example of a 2-3-1 Network, using parameters:
  • $I_1, I_2$ are Inputs
  • b: Bias Value
  • [V], [W]: Connecting Weights
  • O: Output

Static vs. Dynamic Neural Networks

• Static Neural Network
  • No error compensation
• Dynamic Neural Networks
  • Error is fed back to the network to modify its architecture and update the weights.

Training of Neural Networks

• Supervised Learning/Learning with a Teacher
  • Network outputs are compared with target values, and the error is calculated.
  • Error is fed back to the network for updating.
• Un-Supervised Learning/Learning without a Teacher
  • Uses competition, cooperation and updating.

Incremental vs. Batch Modes of Training

• Incremental Training/On-Line Training
  • A particular training scenario is passed through the network.
  • Output(s) are calculated.
  • Error is determined by comparing network output with the target(s).
  • The error is propagated to modify the network.
• Batch Training/Off-Line Training
  • A large numbers of scenarios is passed through the network
  • An average error in predictions is determined.
  • The network is updated based on the average error.