Large Language Models (LLMs)

Large Language Models (LLMs) Defined

• LLMs are a type of machine learning model for natural language processing tasks like language generation
• LLMs are language models trained with self-supervised learning on large quantities of text
• They have many parameters
• Generative Pre-trained Transformers (GPTs) are among the largest and most capable LLMs
• Modern models can be fine-tuned for specific tasks or guided by prompt engineering
• LLMs gain predictive abilities with syntax, semantics, and ontologies, but can also inherit inaccuracies and biases

Historical Context

• In the 1990s, The IBM alignment models was first to develop statistical language modeling
• In 2001, a smoothed n-gram model trained on 0.3 billion words achieved state-of-the-art perplexity
• Researchers constructed internet-scale language datasets in the 2000s as internet use became prevalent
• By 2009, statistical language models dominated over symbolic language models as it could ingest large datasets
• Google made its translation service into Neural Machine Translation in 2016, done by seq2seq deep LSTM networks
• Google introduced the transformer architecture that aimed to improve upon 2014 seq2seq technology
• Transformer architecture is based on attention mechanism
• BERT, an encoder-only model became "ubiquitous" in 2018
• After a decline, research into BERT in 2023 was followed by rapid advancements in decoder-only models
• Decoder-only GPT-1 was introduced in 2018
• GPT-2 caught attention in 2019, but OpenAl deemed it too powerful to release publicly fearing malicious use
• GPT-3 in 2020 went a step further and is available only viaAPI
• ChatGPT a consumer-facing browser-based model captured the imaginations in 2022
• OpenAI released the reasoning model OpenAI o1, in 2024, that generated long chains of thought before returning a final answer
• GPT-4 released in 2023, was praised in its increased accuracy and holy grail for its multimodal capabilities

Popular LLMs

• BLOOM, and LLaMA, have restrictions on the field of use but are source-available models
• Mistral 7B and Mixtral 8x7b models by Mistral AI, have the more permissive Apache License
• DeepSeek released DeepSeek R1 in January 2025

Dataset Preprocessing And Tokenization

• As machine learning algorithms process numbers rather than text, the text must be converted to numbers
• A vocabulary is selected to begin, then integer indices are assigned, an embedding is associated to the integer index
• byte-pair encoding (BPE), and WordPiece are types of algorithms to perform dataset conversion
• "[MASK]" for masked-out token used in BERT
• "[UNK]" ("unknown") for characters not appearing in the vocabulary are special tokens
• Special symbols such as "Ġ" and "##"are used to denote special text formatting
• GPT-3 (Legacy) uses a BPE tokenizer would split tokenizer
• The BPE token split will texts -> series of numerical "tokens" as token izer : texts > series of numerical "tok ens "
• Tokenization also compresses the datasets
• LLMs generally require input to be an array that is not jagged, the shorter texts must be "padded"
• Byte pair encoding merges the pairs of adjacent characters

Data and Model Refinement

• After trained, any text can be tokenized, as long as it does not contain characters that appear in the uni-grams initial-set
• a token vocabulary uses as few tokens as possible for an average English word depending on the frequencies extracted
• Languages such as Portuguese and German have "a premium of 50%" compared to English.
• In a English-optimized tokenizer word in another language is "split" by an amount tokens
• In the context of training LLMs, datasets are typically cleaned by removing low, duplicated, or toxic data
• Datasets are usually clean to increase training efficiency
• Microsoft's Phi series of LLMs is trained on textbook-like data generated by another LLM
• Reinforcement learning from human feedback (RLHF) through algorithms, such as proximal policy optimization, is used to further fine-tune a model based on a dataset of human preferences

Approaches to AI Model Training

• "Self-instruct" approaches to the AI model have been able to bootstrap correct responses
• The largest LLM may be too expensive to train and use directly
• Mixture of experts (MoE) can be applied, pursued by Google researchers since 2017 to train models, to address this cost
• "Soft" weights is achieved via prompt engineering
• Prompts and multiple heads can have varying relevancy

Components of an LLM

• Attention mechanism calculates “soft” weights towards tokens by embedding
• The GPT-2 model has had twelve attention heads and a context window
• Google's Gemini 1.5 has a context window sized up to 1 million
• Anthropic's Claude 2.1, has a context window of up to 200k tokens

Context Limitations and Pre-Training

• Models may be pre-trained either to predict how the segment continues, or what is missing in the segment from its training dataset
• Regularization loss is used to stabilize training during auxiliary tasks
• Infrastructure has substantial bearing on the development of AI models

LLMs in terms of training cost and parameter counts

• GPT-1 of 2018 has only 0.117 billion parameters
• The qualifier "large" in "large language model" is inherently vague
• Training of the GPT-2 (i.e. a 1.5-billion-parameters model) in 2019 cost $50,000
• The PaLM (i.e. a 540-billion-parameters model) in 2022 cost $8 million
• Megatron-Turing NLG 530B (in 2021) cost around $11 million
• For Transformer-based LLM, training costs 6 FLOPs, whereas inference costs 1 to 2 FLOPs

Tool Integration and Agents

• Certain tasks require the use of external tools/software as LLMs cannot solve the use of these tools
• Retrieval-augmented generation (RAG) enhances LLMs by document retrieval systems:
  • A document retriever retrieves relevant documents given a query
  • The model encodes vectors to find the documents with closest vectors of the query
  • LLM then generates an output based on a query of context
• Integration modules transforms an LLM to an autonomous agent that uses dynamic environments, recalls past behaviors, and plans future actions
• The ReAct pattern constructs a LLM with the "Reason + Act" portmanteau
• "Describe, Explain, Plan and Select" (DEPS) is a method where LLM produces plans based around image descriptions

Advanced Tools

• The Reflexion method constructs an agent that learns over multiple episodes:
  • At the end of each episode the LLM thinks up of lessons learned
  • Then are agent in the episodes
• Open - ended exploration, an LLM can be used to score for their "interestingness"
• LLM planners then construct "skills", or "functions" for sequences to store and invoke later

Memory and compression

• LLM-powered agents keep long-term of previous contexts, retrieve, interact socially
• LLMs are trained using or half-precision floating point numbers (float32 and float16).
• Largest models typically have 100 billion parameters, requiring 200 gigabytes to load outside of consumer electronics range(2GB)
• Post-training quantization cuts "space requirement"
• Compression consists of single precision floating point numbers

Multimodality and LLMs

• Multimodality allows for several modalities means several types of input/output, such as video, image, audio, text, proprioception, etc -AI models are trained to ingest "one modality and output another modality"
• AlexNet for image to label, and speech recognition for speech to text are trained to utilize multimodal capabilities
• Common method to create multimodal models out of a LLM is to "tokenize"
• Can construct an LLM that can understand images:
  • Take a trained LLM
• Take
• A trained image encoder can create a small multilayered perceptron that has same dimensions as encoded token

Reasoning Skills

• A direction emerged in LLM development with "reasoning models" train to spend time generating step-by-step resolutions before providing OpenIA with o1 in Sept 2024, and 03 Dec 2024,
• Reasoning models show significant improvements in coding, mathematics, science GPT-4o achieved 13% accuracy while o1 reached 83% on Olympiad exams
• The DeepSeek-R1 Is more cost-effective to operate
• Reasoning models require more resources, have superior capabilities +domains

Hallucinations

• In order to assist with AI and hallucinations, automated reasoning and (retrieval-augmented generation) is used.

Model Properties Depend On

• Cost of pretraining is all cost for the total of calculate.
• The Scaling Laws show LLM performance

Scaling laws

• Scaling laws is the cost of model training.
• Number of parameters is statistical hyper-parameter

Emergent Abilities

• The data indicates what they do as the model becomes more complex
• Balance, logical processing/cognitive shortcuts to adapt to strategies of accuracy/effort

Measuring understanding

• Transcoders, which are more interpretable than transformers, been to develop "replacement models.”
• Technique as partial dependency plots, Shapley Additive exPlanations allows to visualize features
• The small transformers were trained using "the modural arithemetic all addition"
• (whether they are believed to be simply predict, be to plan ahead)
• A Microsoft team argued in 2023, that GPT-4 "can solve novel and difficult tasks spanning (math coding, vision, medicine, psychology +more)"

Evaluation Metrics

• It may produce outputs and what roles characteristics are generated from a model based on traditional gender norms
• Select is referring more to how large the model is to its a token
• It systematically favors certain political viewpoints and ideologies during elections and coverage

Problems During LLM Training

• LLM models may exhibit and learn the training data
• Test Evaluation is is on problem with the larger models, "to give more accurate parts on text"
• Bits words is the Entropy from "quantify common".
• More measure BPT during words and tokens.
• Evaluate is on model as of capacity for the compresser.