Lecture Presentation_Intro to NN.pdf

INTRODUCTION TO NEURAL NETWORKS PERCEPTRONS Fundamentals of Neural Networks, Instr. Kamil Yurtkan 1  Block diagram representation of nervous system:  Neurons encode their out...

INTRODUCTION TO NEURAL NETWORKS PERCEPTRONS Fundamentals of Neural Networks, Instr. Kamil Yurtkan 1  Block diagram representation of nervous system:  Neurons encode their outputs as a series of brief electrical pulses.  These pulses are commonly known as action potential or spike (firing). Fundamentals of Neural Networks, Instr. Kamil Yurtkan 2  Neurons receive input(electric) signals from other neurons through dendrites, integrate them (sum), and generates its own signal which travels along the axon.  The axon, in turn, makes contact with the dendrites of other neurons; thus, the output signal of one neuron becomes input to other neurons (parallelism).  The points of electric contact between neurons are called synapses. Fundamentals of Neural Networks, Instr. Kamil Yurtkan 3 Fundamentals of Neural Networks, Instr. Kamil Yurtkan 4  Cytoarchitectural map of the cerebral cortex. The different areas are identified by the thickness of their layers and types of cells within them. Some of the key sensory areas are as follows: Motor cortex: motor strip, area 4; premotor area, area 6; frontal eye fields, area 8. Somatosensory cortex: areas 3, 1, and 2. Visual cortex: areas 17, 18, and 19. Auditory cortex: areas 41 and 42. (From A. Brodal, 1981; with permission of Oxford University Press.) Fundamentals of Neural Networks, Instr. Kamil Yurtkan 5  Human brain is the accepted model for the artificial neural networks.  When a child borns, the number of neurons are fixed in general, and during the lifetime, the connections between the neurons are changed, developed.  Neural networks are adapted mathematical systems from the nervous system of the brain. Fundamentals of Neural Networks, Instr. Kamil Yurtkan 6  Nowadays, there are many applications using artificial neural networks for problem solving.  Scientific research are highly included in neural networks, publishing journals for the neural networks field of science.  Computer Engineering, Electrical and Electronic Engineering, Industrial Engineering and all other Engineering research fields are highly included with the applications. Fundamentals of Neural Networks, Instr. Kamil Yurtkan 7  Journals:  Neural Networks  Neural Computation  IEEE Transactions on Neural Networks  Network: Computation in Neural Systems  Neurocomputing  Connection Science  International Journal of Neural Systems  Neural Processing Letters  Neural Network Review  Journal of Neural Network Applications  Artificial Life  Neurolinguistics  Cognitive Brain Research  Neural Computing and Applications  Journal of Computational Neuroscience Fundamentals of Neural Networks, Instr. Kamil Yurtkan 8  The main objective is to construct systems which can demonstrate human-like performance in information processing.  Are the computers capable of showing such performance?  All the computers are designed in accordance with the Touring Machines(TM).  Touring Machines can simulate all the decidable problems.  Developing strict algorithms for problem solving is very critical for a computer system (e.g. Sorting several numbers). Fundamentals of Neural Networks, Instr. Kamil Yurtkan 9  Computers are powerful but we have to program them. They do not learn themselves and are not adaptive. NO INTELLIGENCE.  Classical Artificial Intellingence do not bring intelligence, we still need to program them.  On the other hand, human brain is not programmed, even though some genetic code is inherited.  We learn by experience and make decisions in accordance with (e.g. child playing and affected by a fire won’t try it again). Fundamentals of Neural Networks, Instr. Kamil Yurtkan 10  Neural Networks field of science is a multi- disciplinary subject:  Neurobiology  Psychology  Maths  Statistics  Physics  Computer Science  Information Theory  Electrical Engineering Fundamentals of Neural Networks, Instr. Kamil Yurtkan 11  Formal Neuron: w1 x1 x0 w0 x2 w2. Σ+Threshold Activation.. 1 if y >0 -1 otherwise y = Σ wixi xn wn Fundamentals of Neural Networks, Instr. Kamil Yurtkan 12  Basic unit in a neural network  If y=+1 we say that the neuron is firing  A formal neuron with a learning algorithm is called Perceptron.  Linear separator  Parts  N inputs, x1... xn  Weights for each input, w1... Wn  Weighted sum of inputs, y = w0x0 + w1x1 +... + wnxn  A threshold function, i.e 1 if y > 0, -1 if y