Conversational AI and Chatbot Systems

Study Notes

Conversational AI and Chatbot Systems

Conversational AI is a collective term for technologies that enable conversational agents to interact with humans through natural language.
Conversational AI requires rapid processing (less than 300 milliseconds) for a seamless user experience.
The conversational AI pipeline involves three stages: Automatic Speech Recognition (ASR), Natural Language Processing (NLP) or Natural Language Understanding (NLU), and Text-to-Speech (TTS) with voice synthesis.
ASR converts human voice input into readable text. Deep Learning (DL) models like those from Google Cloud, OpenAI, Amazon, and NVIDIA are commonly used.
The ASR process includes feature extraction using MFCCs (Mel Frequency Cepstral Coefficients) and converting audio to Mel spectrograms. Acoustic modeling estimates character probabilities at each time step using extensive datasets (LibriSpeech, Wall Street Journal, Google Audio). Finally, decoding and language processing transform characters into words and phrases, adding punctuation, and preparing the text for further processing.
NLU involves processing and interpreting human language to generate intelligent responses. Its goal is to extract structured information from user messages, including intents and entities.
NLU uses a pipeline architecture: text is converted to tokens, then features, then entities are extracted and intents are classified.
Dialogue Management (DM) controls the next action the assistant takes by considering conversation history and using decision policies like RulePolicy, MemoizationPolicy, and TEDPolicy.
Natural Language Generation (NLG) generates responses using rule-based, retrieval-based, or generative approaches.
Core concepts of conversational agents detail the elements including intents, entities, and actions.
Intents represent the goal of user messages. Entities are extractable data points from user messages. Actions are predicted behaviors the conversational agent takes.
Domains define the knowledge base of the assistant and include responses, intents, slots, and entities.
Stories are structured datasets that train chatbots to manage dialogues. These include user inputs, chatbot reactions, chatbot actions, and entities.
Text-to-speech (TTS) converts processed text into natural-sounding speech using synthesis networks (like Tacotron2) to convert text into spectrograms and vocoders (like WaveGlow) to convert spectrograms to audible waveforms.
Various synthesis models exist, like Tacotron2, GlowTTS, FastPitch, MelGAN, HiFiGAN, SqueezeWave, UniGlow, and FastPitch_HifiGan_E2E. The FastPitch framework uses a feed-forward transformer for enhanced speed.

ASR (Automatic Speech Recognition)

ASR takes human speech and generates text.
Advances in deep learning have improved accuracy in phoneme identification.
Popular DL models include Google Cloud's Speech-to-Text, OpenAI's Whisper, Amazon's Speech Foundation Model, and NVIDIA's Parakeet-TDT.
ASR uses MFCCs to isolate audio features from background noise and convert audio to Mel spectrograms.
Acoustic models employ DL to predict character probabilities using datasets like LibriSpeech, Wall Street Journal, and Google Audio.
Decoding and language processing transform characters into words and phrases.
Word Error Rate (WER) is a measure of ASR accuracy.
Neural networks are used to improve ASR accuracy compared to traditional N-gram models.
Short-Time Fourier Transform (STFT) analyzes audio to identify frequency and phase changes.
Mel spectrograms are used to visually represent frequencies in a signal over time.

NLP (Natural Language Processing)

NLP processes and interprets human language.
It aims to extract structured information (intents and entities) from user messages.
Its primary tasks include Natural Language Understanding(NLU) and Natural Language Generation(NLG).
The architecture is typically a pipeline: processing raw input to generate structured data.

Dialogue Management

Dialogue Management controls the assistant's response based on the conversation history.
Decision policies like RulePolicy, MemoizationPolicy, and TEDPolicy are used to decide on the next action for the most suitable response.