Agentic Context Engineering
Agentic Context Engineering (ACE) is a framework for scalable and efficient context adaptation in large language models (LLMs) that lets an AI system improve itself by treating its own context as an evolving
Machine Learning
Explore learning methods, model architectures, datasets, and practical applications.
Explore articles
Browse subtopics: Natural Language Processing
Articles that also belong to these categories. Counts cover all of Machine Learning.
All articles (858)Deep Learning (187)Natural Language Processing (92)Statistics (90)Training & Optimization (70)Data & Datasets (65)Neural Networks (58)Reinforcement Learning (58)Model Evaluation (55)Artificial Intelligence (54)Large Language Models (49)Mathematics (48)Computer Vision (41)AI Ethics (35)AI Infrastructure (29)AI Safety (26)Open Source AI (25)Data Science (24)AI Benchmarks (22)Developer Tools (20)Algorithms (18)AI Hardware (17)AI Tools & Products (17)Google (16)Generative AI (14)Information Retrieval (14)Model Architecture (13)People (13)AI Agents (12)Interpretability (12)Robotics (12)AI Companies (11)Transformer Models (11)AI Inference (10)MLOps (10)AI Models (8)Computer Science (8)AI Code Generation (7)AI Research (7)Multimodal AI (7)Speech & Audio AI (7)Chinese AI (6)AI Policy & Regulation (5)Meta AI (5)Programming Languages (5)AI History (4)AI for Science (4)Prompt Engineering (4)AI Alignment (3)AI Events (3)Reasoning Models (3)Embodied AI (2)Software Development (2)2025 in artificial intelligence (1)Anthropic (1)Diffusion Models (1)Drug Discovery (1)Education AI (1)Enterprise AI (1)Healthcare AI (1)Image Generation (1)NVIDIA (1)Small Language Models (1)World Models (1)
Showing 1-60 of 92 articles
Attention Is All You Need
"Attention Is All You Need" is the 2017 research paper that introduced the transformer, the neural network architecture that underpins virtually every modern large language model.
BERTScore
BERTScore is an automatic, reference-based metric for evaluating text generation that scores a candidate sentence against one or more references by comparing the contextual embeddings of their tokens rather…
BIG-Bench Hard
BIG-Bench Hard (BBH) is a suite of 23 challenging tasks drawn from the BIG-Bench benchmark, selected because they are "the [tasks] for which prior language model evaluations did not outperform the average…
BLEU (Bilingual Evaluation Understudy)
BLEU (Bilingual Evaluation Understudy) is an automatic evaluation metric that scores the quality of machine translation output by measuring how many word sequences (n-grams) it shares with one or more human…
Bag of Words
The bag of words (BoW) model is a text representation method that converts a document into a fixed-length numeric vector by counting how often each word from a predefined vocabulary appears, while discarding…
Bigram
A bigram (also written 2-gram) is a contiguous sequence of two adjacent elements, typically two words or two characters, taken from a body of text or speech.
Byte-Pair Encoding
Byte-pair encoding (BPE) is a subword tokenization algorithm that splits text into tokens by starting from individual characters or bytes and iteratively merging the most frequent adjacent pair into a new token
CRUXEval
CRUXEval (Code Reasoning, Understanding, and eXecution Evaluation) is a benchmark designed to measure how well large language models can reason about, understand, and mentally execute short Python programs.
Candidate Sampling
Candidate sampling is a family of training-time optimization techniques used in machine learning to reduce the computational cost of models that must choose among a very large number of output classes.
Causal Language Model
A causal language model (CLM), also called an autoregressive language model or a decoder-only language model, is a language model that predicts the next token in a sequence using only the tokens that precede…
Chain-of-Thought
Chain-of-thought (CoT) is a sequence of intermediate steps that a language model generates before its final answer.
Claude Sonnet 4.5
Claude Sonnet 4.5 is a multimodal large language model (LLM) developed by Anthropic and released on September 29, 2025, which Anthropic described at launch as "the best coding model in the world." It is a…
Common Crawl
Common Crawl is a nonprofit 501(c)(3) organization that maintains a free, open repository of web crawl data, and it is the single largest publicly available source of text used to train large language models.
Computer-use agent
A computer-use agent (CUA) is a category of AI agent in artificial intelligence that performs tasks by directly operating a general-purpose computer's graphical user interface (GUI) the way a human does, by…
Conditional Random Field
A conditional random field (CRF) is a discriminative probabilistic model for structured prediction, used most often to assign a label to every position in an input sequence.
Context window
A context window is the finite token sequence that a language model can process for one invocation.
Coreference Resolution
Coreference resolution is the natural language processing task of finding all the expressions in a text that refer to the same entity and grouping them together.
Cosine similarity
Cosine similarity is a measure of similarity between two non-zero vectors that calculates the cosine of the angle between them, defined as the dot product of the vectors divided by the product of their…
Crash Blossom
A crash blossom is a newspaper headline that is unintentionally ambiguous because its compressed wording allows more than one valid parse, producing an unintended (and often comical) alternative reading.
DROP (Discrete Reasoning Over Paragraphs)
DROP** (Discrete Reasoning Over Paragraphs) is a reading comprehension benchmark that requires artificial intelligence systems to perform discrete reasoning operations over textual content.
DSPy
DSPy (short for Declarative Self-improving Python) is an open-source framework, developed at Stanford NLP, for programming rather than prompting large language models (LLMs).
Decoding strategies
Decoding strategies are the algorithms that select output tokens from a language model's next-token probability distribution during text generation.
Direct Preference Optimization (DPO)
Direct Preference Optimization (DPO) is a method for aligning large language models with human preferences that replaces the multi-stage reinforcement learning from human feedback (RLHF) pipeline with a single…
Embedding Layer
An embedding layer is a neural network component that acts as a trainable lookup table, mapping discrete integer indices (such as word IDs, user IDs, or category codes) to dense, continuous-valued vectors.
Embeddings
In machine learning, an embedding is a learned representation that maps an input into a vector, usually so that a downstream model or comparison rule can use relationships encoded in the vector's geometry.
FineWeb
FineWeb is a large-scale, open pretraining dataset for large language models (LLMs) created by Hugging Face.
Full Softmax
Full softmax (also called the standard softmax or exact softmax) is the softmax computation that calculates a probability for every possible output class in a classification model
GLUE benchmark
The General Language Understanding Evaluation (GLUE) benchmark is a collection of nine natural language understanding (NLU) tasks designed to evaluate and compare the performance of language models across a…
Hallucination
Hallucination in generative AI is the production of content that is unsupported, contradicted by an applicable source, factually wrong, internally inconsistent, or otherwise presented without an adequate basis.
In-Context Learning
In-context learning (ICL) is the ability of a large language model to learn a new task at inference time by conditioning on a prompt that contains a few input-output examples (demonstrations)
Information Retrieval
Information retrieval (IR) is the study and engineering of systems that identify material likely to satisfy an information need.
Instruction Tuning
Instruction tuning is the post-pretraining training stage in which a large language model (LLM) is fine-tuned on a curated collection of (instruction
Knowledge Editing
Knowledge editing (also called model editing) is a family of techniques for updating or correcting specific factual associations stored in the weights of a trained large language model without full retraining…
Language Model
A language model is a model of patterns in language that assigns probabilities or comparable scores to linguistic sequences. Its units may be characters, words, subwords, bytes, or other tokens.
Large Language Model
A large language model (LLM) is an artificial intelligence system built on a transformer neural network with billions to trillions of parameters, trained on massive text corpora to predict the next token in a…
Latent Dirichlet allocation
Latent Dirichlet allocation (LDA) is a generative probabilistic model that discovers the hidden thematic structure in a collection of documents by treating each document as a mixture of a small number of…
LiveBench
LiveBench is a contamination-limited benchmark for large language models (LLMs) that draws frequently updated questions from recent sources (math competitions, arXiv papers, news articles, and datasets) and…
LoRA (Low-Rank Adaptation)
Low-Rank Adaptation, usually abbreviated LoRA, is a parameter-efficient fine-tuning method for adapting a pre-trained model.
Low-Resource Languages
Low-resource languages, in natural language processing (NLP), are languages with little of the raw material that modern language technology depends on: digitized text, parallel translation corpora, annotated…
METEOR (metric)
METEOR (Metric for Evaluation of Translation with Explicit ORdering) is an automatic evaluation metric for machine translation and other text-generation tasks that scores a candidate sentence against one or…
Machine Translation
Machine translation (MT) is the automatic translation of text or speech from one human language into another.
Machine learning terms/Natural Language Processing
Natural Language Processing (NLP) is the subfield of artificial intelligence and machine learning concerned with enabling computers to read, interpret, generate, and reason about human language in text and…
Masked Language Model
A masked language model (MLM) is a language model trained to predict missing tokens that have been hidden in a sequence of text, using context from both the left and the right of each blank.
Matryoshka representation learning
Matryoshka Representation Learning (MRL) is a representation learning technique that trains a single neural model to produce embedding vectors which remain useful when truncated to many smaller dimensionalities
Meta Prompting
Meta prompting (also spelled meta-prompting) is an advanced prompt engineering technique where large language models (LLMs) are used to generate, refine, critique, select, or optimize prompts for themselves or…
Multimodal Model
A multimodal model is a machine learning model, or a model-based system, that processes, relates, or produces information across more than one kind of data. Each kind is called a modality.
N-gram
An n-gram is a contiguous sequence of n items extracted from a given sample of text or speech, where the items can be characters, syllables, words, or other linguistic units.
Named entity recognition
Named entity recognition (NER) is the natural language processing task of locating spans of text that name real-world things, such as people, organizations, and locations, and classifying each span into a…
Natural Language Processing
Natural language processing (NLP) is the study and development of computational methods for analyzing, representing, retrieving, transforming, and generating human language.
Natural Language Understanding
Natural language understanding (NLU) is the branch of artificial intelligence and computational linguistics that enables machines to read human language and extract its meaning, intent, and structure.
Natural language generation
Natural language generation (NLG) is the subfield of natural language processing and artificial intelligence concerned with building systems that produce understandable text in English or other human…
Neural Machine Translation
Neural machine translation (NMT) is an approach to machine translation in which a single artificial neural network, trained end to end on bilingual text
Next-token prediction
Next-token prediction is the training objective used by most modern language models: the model reads a prefix of tokenized text, outputs a probability distribution over which token comes next, and training…
OCR Models
OCR Models are artificial intelligence (AI) systems that convert images of typed, handwritten, or printed text into machine-readable digital text through Optical Character Recognition (OCR).
Perplexity
Perplexity has two distinct meanings in artificial intelligence. In information theory and natural language processing, perplexity (PPL) is an intrinsic score for a probabilistic model: on a specified sequence
Post-training
Post-training is the stage of large language model (LLM) development that comes after pre-training and turns a raw, general-purpose base model into an aligned, instruction-following AI assistant.
Pre-Trained Model
A pre-trained model is a machine learning model that has already been trained on a large, general-purpose dataset and can then be reused, either as a fixed feature extractor or by fine-tuning
Pre-training
Pre-training is a stage of machine learning in which a model learns parameters from a source dataset or source objective before those parameters are reused or adapted for a target use.
Prompt Engineering
Prompt engineering is the systematic design and testing of the inputs supplied to a generative model so that the model is more likely to produce a useful result.