Introduction to Deep Learning

• Deep learning is a branch of machine learning and a subset of artificial intelligence.
• Deep learning imitates the human brain using neural networks.
• Deep learning does not require explicit programming.
• Deep learning utilizes nonlinear processing units to perform feature extraction and transformation.
• The output from each layer serves as input for the next layer.
• Deep learning models can identify accurate features with minimal programmer guidance.
• Deep learning algorithms are useful for solving problems of dimensionality.
• Deep learning aims to build algorithms that mimic the brain.
• Deep learning is implemented with neural networks, inspired by biological neurons (brain cells).
• Deep learning employs statistical machine learning techniques to learn feature hierarchies based on artificial neural networks.
• Deep learning is implemented through deep networks, which are neural networks with multiple hidden layers.

Deep Learning Example

• Input layer receives raw image data.
• The input layer determines patterns of local contrast, differentiating based on colors and luminosity.
• The first hidden layer determines face features, fixating on eyes, nose, lips, etc.
• The second hidden layer identifies the correct face template.
• Additional hidden layers can solve more complex problems

Architectures

Deep Neural Networks

• Neural networks incorporating complexity with multiple hidden layers between the input and output layers.
• Highly proficient in modeling and processing non-linear associations.

Deep Belief Networks

• A class of Deep Neural Networks composed of multi-layer belief networks.

Steps to Perform DBN

• A layer of features is learned from perceptible units using the Contrastive Divergence algorithm.
• The previously trained features are treated as visible units, which perform learning of features.
• The whole DBN is trained when the learning of the final hidden layer is accomplished.

Recurrent Neural Networks

• Permits parallel and sequential computation, similar to the human brain.
• Capable of recalling imperative information related to the received input, making them precise.

Types of Deep Learning Networks

• Feed Forward Neural Network
• Recurrent Neural Network
• Convolutional Neural Network
• Restricted Boltzmann Machine
• Autoencoders

Feed Forward Neural Network

• A type of Artificial Neural Network where nodes do not form a cycle.
• Perceptrons are organized in layers: input layer receives input, output layer generates output.
• Hidden layers are not linked to the outside world.
• Each perceptron in a layer is associated with each node in the subsequent layer and all nodes are fully connected.
• There are no back-loops in the feed-forward network.
• Backpropagation algorithm can be used to update the weight values to minimize prediction error.

Feed Forward Applications

• Data Compression
• Pattern Recognition
• Computer Vision
• Sonar Target Recognition
• Speech Recognition
• Handwritten Characters Recognition

Recurrent Neural Network

• Each neuron in the hidden layers receives input with a specific delay in time.
• It accesses the preceding info of existing iterations.
• Involves knowledge about previously used words to guess the succeeding word in a sentence.
• It shares the length and weights crossways time and does not increase model size with the increase in input size.
• The slow computational speed along with it not contemplating any future input for the current state.

Recurrent Applications

• Machine Translation
• Robot Control
• Time Series Prediction
• Speech Recognition
• Speech Synthesis
• Time Series Anomaly Detection
• Rhythm Learning
• Music Composition

Convolutional Neural Network

• Primarily used for image classification, clustering of images, and object recognition.
• DNNs enable unsupervised construction of hierarchical image representations.
• Deep convolutional neural networks are preferred for achieving the best accuracy.

Convolutional Applications

• Identify Faces, Street Signs, Tumors
• Image Recognition
• Video Analysis
• NLP
• Anomaly Detection
• Drug Discovery
• Checkers Game
• Time Series Forecasting

Restricted Boltzmann Machine

• Neurons in the input layer and hidden layer encompass symmetric connections amid them.
• There is no internal association within the respective layer.
• Boltzmann machines have internal connections inside the hidden layer, but Restricted Boltzmann Machines do not.
• These restrictions enable Boltzmann Machines to train efficiently.

Restricted Boltzmann Machine applications

• Filtering
• Feature Learning
• Classification
• Risk Detection
• Business and Economic analysis

Autoencoders

• A type of unsupervised machine learning algorithm where the number of hidden cells is smaller than the input cells.
• The number of input cells is equivalent to the number of output cells.
• An autoencoder network is trained to display an output similar to the fed input and find common patterns and generalize the data.
• These are used for the smaller representation of the input and help in the reconstruction of the original data from compressed data.
• It necessitates the output identical to the input.
• Encoder converts input data in lower dimensions.
• Decoder reconstructs the compressed data.

Autoencoder applications

• Classification
• Clustering
• Feature Compression

Deep Learning Applications

Self-Driving Cars

• Processes images to determine actions like taking a left, right, or stopping.
• Reduces accidents by processing data and deciding actions accordingly.

Voice Controlled Assistance

• Allowing users to tell Siri what to do; Siri then searches and displays the requested information.

Automatic Image Caption Generation

• Generates caption according to the uploaded image.

Automatic Machine Translation

• Converts one language into another.

Limitations of Deep Learning

• Only learns through observations.
• May comprise biases.

Advantages of Deep Learning

• Lessens the need for feature engineering.
• Eradicates needless costs.
• Easily identifies difficult defects.
• Results in best-in-class performance on problems.

Disadvantages of Deep Learning

• Requires an ample amount of data.
• Is quite expensive to train.
• Does not have strong theoretical groundwork.