RNN and GANs PDF

1 RNN IT461 PRACTICAL MACHINE LEARNING Recurrent Neural Networks (RNNs) 2 u A recurrent neural network (RNN) is a type of artificial neural network which uses sequential data or time series data. u They are distinguished by their...

1 RNN IT461 PRACTICAL MACHINE LEARNING Recurrent Neural Networks (RNNs) 2 u A recurrent neural network (RNN) is a type of artificial neural network which uses sequential data or time series data. u They are distinguished by their “memory” as they take information from prior inputs to influence the current input and output. u They are incorporated into popular applications such as Siri, voice search, and Google Translate. RNNs vs FFNs 3 u RNNs generalize feedforward networks (FFNs) to be able to work with sequential data. u RNNs are a type of neural network that has hidden states and allows past outputs to be used as inputs. At each time step, the RNN takes a weighted sum of the inputs and the hidden state, until the final time step where it calculates and returns a final output.  RNNs vs FFNs 4 This line is called the temporal loop. It is used to indicate that the hidden layer not only generates an output, but that output is fed back as the input into the same layer. Feedforward networks (FFNs) take an input (e.g. an image) and The output of RNNs depend on the immediately produce an output (e.g. a digit class), operating on all prior elements within the sequence. elements of the input simultaneously. RNNs architecture 5 u Recurrent Neural Networks (RNNs) have a specific architecture designed to process sequential data while capturing temporal dependencies. u The basic architecture of an RNN consists of three main components: u Input Layer: This layer receives the sequential input data at each time step. The input is fed into the hidden layer for processing. u Hidden Layer: This layer is the core component of an RNN that maintains a hidden state. The hidden state carries information from previous time steps and allows the network to capture temporal dependencies. At each time step, the hidden state is updated based on the current input and the previous hidden state. The specific update rule depends on the type of RNN architecture being used. u Output Layer: This layer produces the output or prediction based on the current hidden state.  Architecture of RNN 6 Source: Standford.edu Different types of RNNs 7 u One To One: there is only one input and one output. u One To Many: a single input and numerous outputs. u Many To One: a single output is produced by combining many inputs from distinct time steps. u Many To Many: there are many options, such as two inputs yield three outputs.  Different types of RNNs 8 output Input Image Image Sentiment Machine Video Classification Captioning Analysis Translation classification RNN Example 9 u An example RNN with 4-dimensional input and output layers, and a hidden layer of 3 units (neurons). u This diagram shows the activations in the forward pass when the RNN is fed the characters "hell" as input. u The output layer contains confidences the RNN assigns for the next character (vocabulary is "h,e,l,o"). RNN problems 10 u RNN are not able to memorize data for long time and begins to forget its previous inputs (short-term memory problem) u When a new information is added, RNN completely modifies the existing information. u RNN is not able to distinguish between important or not so important information. u To overcome this problems, LSTM and GRUs can be used. Long short-term memory (LSTM) 11 u It is a popular RNN architecture, which was introduced by Sepp Hochreiter and Juergen Schmidhuber as a solution to vanishing gradient problem. u They addressed the problem of long-term dependencies. u LSTMs have “cells” in the hidden layers of the neural network, which have three gates–an input gate, an output gate, and a forget gate. u These gates control the flow of information which is needed to predict the output in the network. “Alice is allergic to nuts. She can’t eat peanut butter.” Long short-term memory (LSTM) 12 u LSTM has a hidden state and a cell state. u Hidden state (H): Short term memory u Cell state (C): Long term memory. u LSTM cell has three gates: u Forget gate determines whether the information coming from the previous timestamp is to be remembered or is irrelevant and can be forgotten. u Input gate determines how much of the input node’s value should be added to the current memory cell internal state. u Output gate determines whether the cell passes the updated information from the current timestamp to the next timestamp. A single LSTM Cell Forget gate decides what is relevant to keep from prior steps. Input gate decides what information is relevant to add from the current step. Output gate determines what the next hidden state should be. Example source 13 Gated Recurrent Units (GRUs) 14 u This RNN variant is similar the LSTMs as it also works to address the short- term memory problem of RNN models. u GRU uses only the hidden state to transfer information. u It also only has two gates, a reset gate and update gate. u The reset gate is used to decide how much past information to forget. u The update gate decides what information to throw away and what new information to add. Applications of RNNs 15 u RNNs have a wide range of applications across various domains, including: u Natural Language Processing (NLP): machine translation, sentiment analysis, text generation. u Speech Recognition: Phoneme recognition, speech-to-text transcription. u Time Series Analysis: Stock market prediction, weather forecasting, energy load prediction. u Image Captioning: Generating textual descriptions for images. u Handwriting Recognition: Converting handwritten text into digital text. u Music Generation: Generating new musical compositions.  16 GAN GENERATIVE ADVERSARIAL NETWORKS Generative Adversarial Networks (GANs) 17 u Generative adversarial networks (GANs) are neural networks that are used primarily for generating synthetic data, such as images, music, speech, or text. u GANs were introduced by Ian Goodfellow and his colleagues in 2014. Descriptive vs. Generative models 18 u Descriptive and generative models are two different approaches within the field of machine learning. u Descriptive models are designed to describe and predict patterns in existing data. u Generative models focus on learning the underlying data distribution to generate new and realistic samples (unsupervised learning).  Descriptive vs. Generative models 19 Descriptive Model Generative Models Purpose Descriptive models aim to describe and Generative models aim to understand and model summarize existing data patterns and the underlying data distribution to generate new, relationships. synthetic data that resembles the original data. Training Descriptive models are typically trained on Generative models are trained on existing data to labeled or unlabeled data to learn patterns and learn the statistical patterns and dependencies make predictions or classifications. within the data. Output The primary output of descriptive models is The primary output of generative models is the predictions or classifications based on the generation of new samples that are similar to the learned patterns in the data. original data distribution. Examples Regression models, decision trees, and SVM. Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and Hidden Markov Models (HMMs). How GANs Work 20 u GANs consist of two components: a generator and a discriminator. u The generator creates synthetic data samples, such as images or text, while the discriminator tries to distinguish between real and generated samples. u The generator and discriminator are trained together in a competitive process, where they learn from each other's feedback.  GAN Architecture 21 o The generator takes a random noise vector as input and generates synthetic data, such as images or text. o The generator typically consists of multiple layers, such as fully connected layers or convolutional layers, which transform the o The discriminator acts as a binary classifier that aims to distinguish input noise into a meaningful output. between real and generated data. o Its goal is to produce data that is o It takes an input (either real or generated data) and predicts the indistinguishable from the real data. probability of it being real. o The discriminator is trained on a dataset of real and generated data and learns to differentiate between real and generated samples. o Its objective is to maximize the probability of correctly classifying real and generated data. GAN Training 22 The training process of GANs involves a competitive game between the generator and discriminator: u The generator and discriminator are initialized with random weights. u The generator generates synthetic data by taking random noise as input. u The generator aims to maximize the probability of the discriminator incorrectly classifying the generated data as real. u The generated data is fed to the discriminator. u The discriminator is presented with a combination of real and generated data. u The discriminator aims to correctly classify the real and generated data. u The generator and discriminator iteratively updating their weights in a competitive manner. u The training process continues until the generator produces data that is difficult for the discriminator to distinguish from real data. u By training the generator and discriminator together, GANs learn to generate synthetic data that captures the characteristics and distribution of the real data. Applications of GANs 23 u GANs have numerous applications across various domains, including: u Image generation and synthesis: generate realistic images that resemble a specific dataset. u Video generation and prediction: generate new video sequences or predict future frames based on the observed frames. u Text-to-image synthesis: generate images based on textual descriptions. u Style transfer and image editing: transfer styles from one image to another. u Data augmentation: generating synthetic samples that expand the diversity and quantity of the dataset. Challenges 24 u GANs are powerful but come with challenges: u Training instability: GANs can be difficult to train and prone to model collapse (i.e. generator produces limited variations, failing to capture the entire data distribution). u Evaluation: It's challenging to evaluate the quality of generated samples objectively. u Computational resources: GANs often require significant computational power and training time. u Ethical considerations: GANs raise concerns about fake media and potential misuse.  25 THE END THANK YOUJ