AI Agents
An AI agent is a software system that selects and performs actions in an environment in pursuit of an objective.
Large Language Models
Explore language models, how they work, and the techniques used to build applications with them.
Explore articles
Browse subtopics: Machine Learning
Articles that also belong to these categories. Counts cover all of Large Language Models.
All articles (545)AI Models (184)Open Source AI (165)Chinese AI (92)Natural Language Processing (58)AI Companies (53)Machine Learning (49)Mixture of Experts (46)Multimodal AI (41)Reasoning Models (39)Artificial Intelligence (37)OpenAI (35)AI Benchmarks (33)Anthropic (31)Google DeepMind (26)AI Agents (22)AI Safety (21)Developer Tools (20)Meta AI (20)AI Code Generation (18)Small Language Models (18)Training & Optimization (18)AI Inference (16)Model Architecture (14)Generative AI (13)NVIDIA (13)Prompt Engineering (13)AI Research (12)Deep Learning (12)Google (12)Model Evaluation (12)Data & Datasets (11)AI Tools & Products (9)Conversational AI (9)Microsoft (9)Transformer Models (9)AI Infrastructure (7)Diffusion Models (7)AI Alignment (5)Enterprise AI (5)Research Organizations (5)AI Hardware (4)Healthcare AI (4)Information Retrieval (4)Interpretability (4)People (4)Mathematics (3)AI History (2)AI for Science (2)Algorithms (2)Computer Vision (2)Legal AI (2)Reinforcement Learning (2)Speech & Audio AI (2)AI Policy (1)AI Policy & Regulation (1)AI in Cybersecurity (1)AI in Gaming (1)Autonomous Vehicles (1)ChatGPT (1)China AI (1)Drug Discovery (1)Neural Networks (1)Robotics (1)Software Development (1)Video Generation (1)
Showing 1-49 of 49 articles
Agentic Context Engineering
Agentic Context Engineering (ACE) is a framework for scalable and efficient context adaptation in large language models (LLMs) that lets an AI system improve itself by treating its own context as an evolving
Autoregressive Model
An autoregressive model predicts each element of a sequence from the elements that precede it, feeding its own earlier outputs back in as context for every later prediction.
BIG-Bench
BIG-Bench (Beyond the Imitation Game Benchmark) is a large-scale, collaborative benchmark of 204 tasks, contributed by 450 authors across 132 institutions, built to measure and extrapolate the capabilities of…
Byte-Pair Encoding
Byte-pair encoding (BPE) is a subword tokenization algorithm that splits text into tokens by starting from individual characters or bytes and iteratively merging the most frequent adjacent pair into a new token
Chinchilla scaling laws
The Chinchilla scaling laws are a set of empirical findings published by DeepMind researchers in 2022 showing that, for a fixed compute budget, a large language model trains most efficiently when its number of…
Claude Sonnet 4.5
Claude Sonnet 4.5 is a multimodal large language model (LLM) developed by Anthropic and released on September 29, 2025, which Anthropic described at launch as "the best coding model in the world." It is a…
Compound AI System
A compound AI system is an AI system that achieves its objectives by combining multiple interacting components, such as large language models, retrieval mechanisms, external tools, guardrails, and…
Context window
A context window is the finite token sequence that a language model can process for one invocation.
DSPy
DSPy (short for Declarative Self-improving Python) is an open-source framework, developed at Stanford NLP, for programming rather than prompting large language models (LLMs).
Decoding strategies
Decoding strategies are the algorithms that select output tokens from a language model's next-token probability distribution during text generation.
Emergent abilities
Emergent abilities are capabilities of large language models (LLMs) that are absent in smaller models but appear once a model reaches sufficient scale.
FineWeb-Edu
FineWeb-Edu is an open, English-language pretraining dataset of roughly 1.3 trillion tokens
GGUF
GGUF (GPT-Generated Unified Format) is the standard binary file format for storing large language models for local inference, bundling a model's weights, tokenizer, and metadata into a single self-contained…
GSM8K
GSM8K (Grade School Math 8K) is an English-language benchmark of grade-school arithmetic word problems released by OpenAI researchers in 2021.
In-Context Learning
In-context learning (ICL) is the ability of a large language model to learn a new task at inference time by conditioning on a prompt that contains a few input-output examples (demonstrations)
Inference optimization
Inference optimization is the set of techniques that make running a trained artificial intelligence model, especially a large language model, faster, more memory-efficient, and cheaper to serve in production.
Instruction Tuning
Instruction tuning is the post-pretraining training stage in which a large language model (LLM) is fine-tuned on a curated collection of (instruction
Knowledge Editing
Knowledge editing (also called model editing) is a family of techniques for updating or correcting specific factual associations stored in the weights of a trained large language model without full retraining…
Large Language Model
A large language model (LLM) is an artificial intelligence system built on a transformer neural network with billions to trillions of parameters, trained on massive text corpora to predict the next token in a…
Llama 2
Llama 2 is a family of open-weight large language models developed by Meta AI. Meta released pretrained and dialogue-tuned checkpoints with 7 billion, 13 billion, and 70 billion parameters on July 18, 2023.
MBPP
MBPP (Mostly Basic Python Problems) is a code generation benchmark of 974 crowd-sourced Python programming tasks designed to be solvable by entry-level programmers, introduced by Jacob Austin, Augustus Odena…
MMLU-Pro
MMLU-Pro (Massive Multitask Language Understanding Professional) is an artificial intelligence benchmark of 12,032 ten-choice questions across 14 academic domains
Machine learning terms/Natural Language Processing
Natural Language Processing (NLP) is the subfield of artificial intelligence and machine learning concerned with enabling computers to read, interpret, generate, and reason about human language in text and…
Mixture of Agents
Mixture of Agents (MoA) is a multi-model collaboration framework that combines multiple large language models (LLMs) in a layered architecture
Model merging
Model merging combines the parameters of multiple trained neural networks into a single unified model without any additional training.
Multi-token prediction
Multi-token prediction (often abbreviated MTP) is a language modeling training objective in which the model is trained to predict several future tokens at each context position rather than only the next token.
NOSA (Native and Offloadable Sparse Attention)
NOSA (Native and Offloadable Sparse Attention) is a trainable sparse attention mechanism designed so that most of a language model's KV cache can live in CPU memory during decoding without the CPU-to-GPU…
Next-token prediction
Next-token prediction is the training objective used by most modern language models: the model reads a prefix of tokenized text, outputs a probability distribution over which token comes next, and training…
ORPO
ORPO (Odds Ratio Preference Optimization) is a preference alignment algorithm for large language models that merges supervised fine-tuning and preference alignment into a single training stage, eliminating the…
Post-training
Post-training is the stage of large language model (LLM) development that comes after pre-training and turns a raw, general-purpose base model into an aligned, instruction-following AI assistant.
Prompt Caching
Prompt caching is an large language model (LLM) inference optimization that stores the computed key-value (KV) state of a repeated prompt prefix so it can be reused across API calls, cutting both cost and…
Prompt Engineering
Prompt engineering is the systematic design and testing of the inputs supplied to a generative model so that the model is more likely to produce a useful result.
Protein language model
A protein language model (pLM) is a neural network trained on raw amino acid sequences using the same self-supervised objectives that power text models, with individual amino acids serving as tokens instead of…
Reasoning (artificial intelligence)
Reasoning in artificial intelligence is the ability of an AI system to draw inferences, solve problems, and make decisions through structured, multi-step thought rather than a single immediate response.
Reasoning models
Reasoning models are large language models that are post-trained or configured to spend additional inference-time computation on intermediate steps before returning an answer.
Retrieval-Augmented Generation
Retrieval-augmented generation (RAG) is a family of methods that retrieves information from an external collection and conditions a generative model on that information when producing an output.
SGLang
SGLang (short for Structured Generation Language) is a high-performance, open-source serving framework for large language models and multimodal models, originally developed at UC Berkeley's Sky Computing Lab…
Scaling Laws
Neural scaling laws are empirical relationships that describe how a measured outcome changes as a neural network, its training data, or the computation used to train it becomes larger.
Scaling Laws for Neural Language Models
Scaling Laws for Neural Language Models is a landmark research paper published by OpenAI on January 23, 2020 (arXiv:2001.08361) that established that the test loss of a neural language model falls as a smooth…
Small language model
A small language model (SLM) is a compact large language model, typically under about 10 billion parameters, built to run efficiently on a single consumer GPU, a laptop, or a smartphone rather than in a data…
Speculative Decoding
Speculative decoding is a lossless inference acceleration technique for autoregressive transformer models in which a small, fast draft model proposes several future tokens at once and the larger target model…
Temperature (artificial intelligence)
Temperature is a hyperparameter that controls the randomness of a large language model's output by scaling the model's raw scores, called logits
Test-time compute
Test-time compute (also called inference-time compute scaling or test-time scaling) is the practice of allocating additional computation while a large language model answers a query
Tokenization
Tokenization is the process of breaking text into smaller units called tokens, which serve as the fundamental input to natural language processing (NLP) systems and large language models (LLMs).
Top-p and top-k sampling
Top-p sampling, also called nucleus sampling, is a text-generation decoding method for large language models (LLMs) that, at each step, samples the next token from the smallest set of highest-probability…
Top-p sampling
Top-p sampling, also called nucleus sampling, is a stochastic decoding method for text generation in which the model samples from the smallest possible set of tokens whose cumulative probability mass exceeds a…
llama.cpp
llama.cpp is an open-source large language model inference engine written in C and C++ by Bulgarian software engineer Georgi Gerganov that runs large language models on consumer-grade hardware without…
vLLM
vLLM is an open-source inference and serving engine for large language models. It is software for running already trained models, either through a Python interface for offline work or through a network server…