Natural Language Processing (NLP)

Natural Language Processing (NLP)

• NLP focuses on enabling computers to understand, interpret, and generate human language in a meaningful way.
• Key areas include text processing, syntax and semantics analysis, named entity recognition (NER), sentiment analysis, machine translation, speech recognition, and text generation.

NLP Use in Generative AI

• NLP plays a key role in generative AI, particularly for generating text and language-based content.
• NLP techniques are used to construct models, generating news articles, stories, poems, and code.
• NLP allows development of chatbots and virtual assistants with intent recognition, response generation, that maintain context in conversation.
• Generative models powered by NLP enable accurate and fluent translations between different languages.
• NLP techniques applied to create generative models summarize long documents, distilling key information.
• NLP adapts content considering user preferences, like personalized news, recommendations, and marketing content.
• It helps writers by suggesting ideas, generating plots, or drafting content.

NLP in Generative AI Tools

• GPT-3 is a language model using NLP to produce human-like text, write essays, answer questions, and generate code.
• BERT, developed primarily for understanding text, can be fine-tuned for generative AI tasks.
• Transformer models form the basis for NLP systems, which are crucial for generative AI, enabling coherent text generation by modelling word dependencies.

AI Systems and Tools

• Alexa and Siri are AI assistants performing tasks, setting alarms, playing music, making calls, and controlling smart home devices.
• Alexa was developed by Amazon and released in 2014, available on devices such as Amazon Echo, Echo Show, and Echo Dot.
• Siri was developed by Apple and released in 2011, available on Apple devices such as iPhone, iPad, Apple Watch, and HomePod.
• Generative AI excels at an entirely new pace, by creating fresh content such as poems, composing music, or generating code.

AI Layers

• Machine Learning (ML) enables software to predict outcomes without explicit programming, using historical data as input.
• Deep Learning (DL), a type of ML, uses artificial neural networks inspired by the human brain to learn from data.
• Generative AI creates content like text, code, images, and music, and models learn from large datasets.
• Large Language Models (LLMs) are generative AI models trained on text and code, which generate text, translate languages, and answer questions.
• Generative Pre-trained Transformers (GPTs) are LLMs using a transformer architecture, well-suited for natural language processing.
• GPT-4 and ChatGPT are examples of GPT models, GPT-4 is an LLM and ChatGPT is designed for chatbots.

Rise of Generative AI

• Generative AI focuses on creating new content such as text, code, images, and music, experiencing rapid growth.
• Algorithms trained on massive datasets fuel advancement, which allows learning of complex patterns and outputting indistinguishable human creations.
• Generative AI models can answer questions, follow commands, and imagine/produce concepts, which gives way to advancement across fields.
• AI-powered tools can write marketing copy, compose tailored music, generate detailed images, and develop innovative applications.

Responsible AI

• Ethical implications, potential biases, and job displacement are important considerations.
• It is key to create responsible frameworks for its development and use.
• The potential of generative AI can be harnessed while mitigating risks.

Machine Learning Types

• AI allows learning and improvement without explicit programming.
• Supervised learning uses labelled data, that enables accurate output predictions for unseen data, for tasks like spam filtering and image recognition.
• Unsupervised learning uses unlabelled data to discover patterns like customer segmentation, fraud detection, and dimensionality reduction.
• Semi-supervised learning combines labelled and unlabelled data, refining the algorithm and expanding knowledge of task at hand.

Common application of Semi-Supervised learning

• Applications of semi-supervised learning are image/text classification, anomaly detection, and recommendation systems.
• Techniques have strengths/weaknesses, the choice relying on the specifics of task plus data availability.

Deep Learning Details

• Capability fuelled by the power of artificial neural networks (ANNs).
• ANNs consist of interconnected nodes, arranged in layers.
• Neurons perform a mathematical operation on its input, with the output passed to other neurons.
• The input layer receives raw data in differing forms
• Neurons perform a calculation on inputs using hidden layers, passing the result to the next layer.
• The output layer produces the final result after a calculation.
• The network learns by adjusting connection weights between neurons, that are updated minimising error.
• Convolutional Neural Networks (CNNs) are well-suited to image/video recognition tasks.
• Recurrent Neural Networks (RNNs) are effective for processing sequential data, natural language and time series data.
• Generative Adversarial Networks (GANs) are used to generate new data like images, music and text.
• Earlier generations of neural networks suffered from a critical limitation being their lack of context.

Details Regarding the Transformer Model

• Transformer responses were pre-programmed, which limits innovation.
• The model emerged as a response to limitations, specifically in Natural Language Processing (NLP).
• The transformer model is built on the concept of attention, allowing it to focus when making predictions.
• The concept captures long-range dependencies plus superior performance on various tasks.

Key Components of Transformer Model

• The encoder translates sequence to series of vector representations, self-attention mechanisms attending to input parts, which allows understanding of input context.
• The decoder uses representation from encoder which generates the output sequence.
• The layers decode self-attention to encoded parts, plus encoder-decoder attention to decoder.
• Multi-head attention operates by attending to differing data aspects at the same time.
• Heads learn to focus on data features, assisting the model when understanding input, allowing it to generate accurate outputs more accurately.
• Positional encoding ensures the model understands relative positions without RNN-style processing.
• Connection type ensures stability and efficiency.

Transformer Addressing of Limitations

• Limitations were addressed through the concept of attention and self-attention for focus when making predictions.
• Improved Encoder-decoder architecture allows efficient processing in parallel.
• Enhanced Positional encoding allows to understand elements in sequence even without explicit RNN-style processing.
• Improved Multi-head attention allow to attend simultaneously to differing aspects when understanding context while improving overall.

Impact

• The model achieved improvements in NLP duties, from machine translations, to document summarization, plus response output.
• Paved the path for GenAI models such as GPT-3 and ChatGPT, delivering human quality responses in varying areas.

Purpose of GenAI

• Subfield within AI focused on creating new and evolving content.
• Unlike traditional AI, trained on underlying models, plus content structures that it aims to create.
• The models harness known data to create outputs that are statistically aligned with the information it trained on.
• Tasks can include expressing a creative writing style in varying forms.
• Allows translation whilst sustaining context.
• Allows the developing, designing and expression of personalised product designs.

Generative AI Fascinating Metrics

• Statista expects the GenAI market to show an annual growth rate (CAGR 2023-2030) of 24.40%, resulting in a market volume of $207 billion by 2030 (Aug 2023).
• The largest market size is expected, is $16.14 billion.
• Chat GPT is considered the the most used text generation tool currently.
• Midjourney is considered the most used Gen AI tool overall.
• The market is posed to become $1.3 Trillion by 2032.
• Top 5 categories for GenAI to be used are across high-tech, banking, pharmaceutical, education and telecommunications.

Working Models

• Process starts with the collection and processing by extensive data, for the Al model to learn and generate content.
• Model training is used to recognise patterns and features within data.
• Pattern learning and understanding is a critical phase which enables AI to develop capabilities, plus a coherent and meaningful response.

Continued

• After establishing patterns for understanding, the initial text, audio, images and code are then generated.
• Lastly final stage requires refining content and iterative use of feedback loops, the process enabling outputs to achieve a higher quality.
• Process requires refinement using improvement loops to ensure output is high quality, while is continuously improved with iterative loops.

Traditional Machine Learning vs GenAI

• Traditional focus on outcomes, GenAI designed to find new patterns from data.
• GenAI excel thanks to the generation of diverse content, analysis and the ability to produce novel aspects.

LLM

• Creates complex relationships in words, plus phrases, supporting language functionality, which allows output of sentences by:
  • The ability to make effective realistic sentences.
  • Allows accurate translations plus informative responses.
  • Provides summarised data.

Transformer based Architecture

• A focus on parts based of input texts allows Generation of data to be produced, from trained language models based in training.
• Data is given massive data to build and enhance relationships for complex functionality.

LLM Examples

• Bard is a Google AI Chatbot from Gemini, enhances searches across all possible available platforms.
• ChatGPT is a variant from Open AI, which can simulate dialogues from customer service.
• DALL-E is a modelling system which has image generation ability.
• Midjourney is a tool which uses language, produces image results based from AI.
• Stable diffusion, creates quality images from high quality prompts on text.

Component Overview

• The models components are composed of data, architecture and training.
• Cornerstone of model is diverse data, extensive in language, patterns and breadth.
• Architecture from underlying training supports the model to connect to the training.
• Training data is based from a model, the language then understands, while being able to achieve the desired output.

LLM Learns

• To understand how training works, need closer understanding based on the inputs provided.
  • Inputs then transform in numerical, vectors in the same space of meaning.
  • Embeddings then get passed from transformer model which are able to capture context.

Human Influence

• The processes provide humans with the ability to provide output using expectations.
• After generations, model uses pre-trained functionality to allow predictions in generating quality.
• Last process delivers models that transfer back results used with accurate training.

LLM Overview

• These high level areas are needed to achieve an outcome.
  • Customise from LLM, make an output to get desired text.
• Set parameters in order to achieve the AI for online tooling using AI tools.

Models

• These tools allow a natural processing of data to read manipulation, in areas such as content.
  • Personalised news and generate stories.
  • Effective and persuasive market writing.
  • Creative writing assists in poems scripts.
  • Autonomously code data, automate assist developers with data entries.
• Education Tools:
  • Enhance personalised tooling.
  • Automated grading for feedback, saving students time and effort.
• Customer Support:
  • AI tooling, allows resolution and effective support with quality.
  • Automated responses saves time, automates time.
  • Enhancing with social feedback and gaining quality insights.
• Research and Development:
• Accelerates discovery with data amounts.
• Medical diagnosis and treatment, records.
• Media and Communication:
• Personal entertainment with film, music.
• Develops gaming for immersion.