Understanding Artificial Intelligence

• Artificial Intelligence (AI) refers to the ability of a computer or machine to perform tasks or make decisions similarly to humans.
• AI designers aim to replicate human-like attributes, including creativity, logical reasoning, & knowledge acquisition, in systems.
• AI simulates human intelligence processes using machines, particularly computer systems.

Types of Artificial Intelligence

• AI can be categorized based on capabilities or functionalities.
• Capabilities refers to what the AI can do.
• Functionalities refers to how the AI achieves its capabilities, including operation & learning mechanisms.

Types of AI Based on Capabilities

• Narrow AI (Weak AI) are AI systems trained for specific tasks and cannot exceed their programmed abilities, having a "narrow" scope.
• They perform only as designed, with limited range.
• Apple's Siri uses Narrow AI with voice recognition for relevant user responses
• Machine Learning & Deep Learning are examples of Narrow AI.
• General AI (Strong AI) is an AI agent's aptitude to perceive & comprehend information akin to human intelligence.
• These systems can build multiple competencies independently and establish connections across domains,
• This reduces training time, making AI systems as capable as humans by replicating capabilities.
• Super AI (Artificial Super Intelligence or ASI) is AI research's pinnacle.
• It can replicate human dynamic intelligence and emulate tasks with greater memory, faster data analysis/processing, & advanced decision-making.
• ASI understands human sentiments and experiences and evokes emotions, but its is hypothetical and credibility is debated.

Types of AI Based on Functionalities

• Reactive Machines are the oldest AI systems with very limited capabilities.
• They emulate the human mind's response to stimuli.
• These machines lack memory-based functionality, preventing them from using past experiences to inform current actions, thus, they cannot "learn".
• Reactive Machines respond automatically to a limited set/combination of inputs and cannot rely on memory.
• IBM's Deep Blue, which defeated chess Grandmaster Garry Kasparov in 1997, exemplifies a reactive AI machine.
• Limited Memory machines learn from historical data to make decisions.
• Present-day AI systems using deep learning are trained on large volumes of stored data, forming a reference model for future problem-solving.
• Image recognition AI is trained with images and labels to name scanned objects to teach the AI.
• When an image is scanned, the AI uses training images as references to understand the contents and labels new images with increasing accuracy due to learning experience.
• Most present-day applications, from chatbots & virtual assistants to self-driving vehicles, are driven by limited-memory AI.
• Theory of Mind AI is under development, aiming to understand entities by discerning their needs, emotions, beliefs, & thought processes during interaction, unlike previous AI types.
• Artificial emotional intelligence is a growing industry, and theory of mind AI progress requires the development of more branches of AI.
• Self-aware AI is a theoretical future stage where AI has evolved to mirror the human brain, developing self-awareness.
• This AI understands and evokes emotions and possesses its own emotions, needs, beliefs, and potential desires.

AI Sub-Disciplines

• AI's sub-disciplines include Machine Learning, Computer Vision, Cognitive Computing, Natural Language Processing, Neural Networks & Deep Learning

Biological Neural Networks (BNN)

• BNN is the area in the human brain that enables actions.
• The neuron cell is a key component in the human neural system
• A neuron is seen as a small processor and memory unit in the human brain.
• The brain is a complex, non-linear system that processes information in parallel.
• The brain handles recognition, perception, & motor control faster than digital computers.
• BNN is characterized by robustness, fault tolerance, flexibility, adaptability to new environments through learning, ability to handle fuzzy/noisy/inconsistent information, high parallelism, small size, compactness, & low power consumption.
• The human brain consists of approximately 10^11 neurons.
• Neurons communicate via axons
• Axons split into smaller fibers linked to other neurons via synapses.
• Synapses connect dendrites to the neuron's main body (Soma).

How Neurons Work

• Synapses function as one-way valves.
• An electrical signal is generated by the neuron, passing down the axon to synapses.
• Synapses connect onto other neuron dendrites.
• Electrical signals trigger the release of transmitter chemicals that flow across the synaptic cleft.
• Chemicals can have an excitatory effect making the receiving neuron more likely to fire.
• Chemicals can have an inhibitory effect making the receiving neuron less likely to fire.
• A neuron fires when the sum of inhibitory and excitatory inputs exceeds its threshold.

Biological Neural Network Actions

• BNN allows humans to do Classification where they classify inputs into groups
• BNN allows humans to do Clustering where they group similar patterns together
• BNN allows humans to do Mapping by associating pattern with the information

Learning in Neural Networks

• Knowledge in neural networks is represented by the strength of synaptic connections between neurons, known as "connectionism."
• Learning is achieved by adjusting synaptic strengths (weights).

Types of Learning Algorithms

• Supervised learning uses known input-output pairs for training.
• Reinforcement learning uses a single good/bad training signal.
• Unsupervised learning occurs without a training signal; self-organization and clustering are produced.

Artificial Neural Networks (ANN)

• ANNs process information with characteristics similar to biological neural networks.
• ANNs developed from generalizations of mathematical models related to human cognition or neural biology.
• Information processing occurs at neurons.
• Signals travel across connection links between neurons.
• Each connection link has a weight that multiplies the transmitted signal.
• Net input is the weighted sum of the input signals.
• Each neuron applies a transfer activation function to its net input to determine the output signal.
• Neurons have a single threshold value.
• Output signal is discrete (e.g., 0 or 1) or a real-valued number (e.g., between 0 and 1).

Artificial Neurons

• They are basic building blocks to construct complicated networks
• Neurons are computational units performing calculations based on connected units.

Characteristics of Artificial Neurons

• Bias values shift the activation function.
• Bias values do not interact with input data.
• Weights alter the activation function's steepness.
• A linear neuron becomes more flexible with a bias.
• A bias unit always has an output value of 1.
• Bias is connected to hidden & output layer units via modifiable weights.
• Adding a bias helps convergence of the weights.
• Bias is equivalent to a weight on an extra input line having an activity of 1.

Characteristics of ANNs

• Architecture refers to the arrangement of nodes and connections, i.e., structure.
• Training/learning algorithm is the method for setting the weights of connections.
• Activation function produces an output based on node inputs.

Architecture of ANNs

• Architecture involves arranging neurons into layers with defined connection patterns.
• Feed-forward networks (FFN) pass connections in a single direction
• Feed-Forward NNs include single-layer and multi-layer networks
• Feed-backward networks (recurrent) include connections from later to previous layers.
• Associative networks lack a hierarchical structure

Feed Forward NN

• Neurons are arranged in separate input, hidden, and output layers.
• There isn't connection between neurons in the same layer
• Neurons receive inputs from previous layers.
• Neurons that send a signal in one layer deliver an output to the next layer.
• Signals are only allowed to travel one way from input to output.
• Allows no feed back
• Associates input with output
• Connections are unidirectional (hierarchical).
• Single-layer networks have one layer of connection weights.
• The input layer is connected to output layer neurons.
• Input neurons are fully connected to output units, not other input units.
• Output neurons aren't connected to other output units.
• Multi-layer networks contain one or more layers of hidden neurons between the input output layers.
• There's a layer of weights between two adjacent levels of units (input, hidden, or output).
• They solve more challenging problems, but training is more difficult.

Feed-Back (Recurrent) NN

• Some connections extend from a layer to previous layers.
• More biologically realistic.
• Signals travel in both directions by introducing loops.
• Powerful and complicated.
• Dynamic network
• has at least one feedback loop

Associative Network

• Lacks a hierarchical structure, with bidirectional connections.

Training/Learning Algorithms

• Training involves modifying system behavior by adjusting parameters due to environmental simulation.

Elements of Learning

• Involves simulation from the environment
• Follows parameter adaptation rules
• Leads to change of behavior.
• NN learning adapts the free parameters of a neural network through environmental simulation during the embedding process.

Types of Training

• Supervised learning provides inputs with target outputs.
• The network adjusts weights to produce an output close to the target signal with weights
• A common method is error correction, using Least Mean Square (LMS) or Back Propagation.
• Unsupervised learning presents inputs with a teacher signal target
• Algorithms adjust with weights
• A common method is error correction, using Least Mean Square (LMS) or Back Propagation.
• Reinforcement training involves generalizing from supervised learning.
• It uses random searches until the correct answer is found.
• The teacher scores training example performance.
• Actions based on performance scores result in random weight adjustments.
• Supervised learning associates each pattern with a target output vector (input, desired output).
• It solves problems like classification, regression, and pattern recognition.
• Models used are perceptron and Heb.
• Unsupervised learning involves no target output vector (different input).
• Problems are clustering and data reduction.
• Models used are self-organizing maps (SOM) and Hopfield.