Discrete diffusion language model
A discrete diffusion language model is a class of generative model for text that produces tokens by iteratively denoising a corrupted sequence, rather than by predicting one token at a time from left to right.
Natural Language Processing
Explore Natural Language Processing through related topics and the articles other pages reference most.
Explore articles
Browse subtopics (51)
Articles that also belong to these categories. Counts cover all of Natural Language Processing.
All articles (239)Machine Learning (92)Large Language Models (58)AI Benchmarks (42)Deep Learning (37)Information Retrieval (29)AI Models (24)Artificial Intelligence (24)Open Source AI (23)Data & Datasets (15)AI Companies (9)Model Evaluation (9)Transformer Models (9)Meta AI (8)Speech & Audio AI (8)OpenAI (7)Training & Optimization (7)AI Ethics (6)AI Safety (6)AI Tools & Products (6)Computer Vision (6)Generative AI (6)People (6)AI Agents (5)Prompt Engineering (5)Developer Tools (4)Neural Networks (4)AI Code Generation (3)Algorithms (3)Anthropic (3)Mathematics (3)Reasoning Models (3)Statistics (3)AI Alignment (2)AI History (2)AI Research (2)Chinese AI (2)Diffusion Models (2)Google DeepMind (2)Healthcare AI (2)Model Architecture (2)Multimodal AI (2)AI Inference (1)Computer Science (1)Education AI (1)Enterprise AI (1)Legal AI (1)Microsoft (1)Mixture of Experts (1)NVIDIA (1)Research Organizations (1)Voice AI (1)
Showing 61-120 of 239 articles
DistilBERT
DistilBERT is a compressed version of BERT released by Hugging Face in October 2019 that is 40% smaller and 60% faster than BERT-base while retaining 97% of its language-understanding performance on the GLUE…
ELECTRA
ELECTRA, which stands for Efficiently Learning an Encoder that Classifies Token Replacements Accurately
ELIZA
ELIZA is the first widely known chatbot, a natural language processing program written by Joseph Weizenbaum at the Massachusetts Institute of Technology (MIT) between 1964 and 1967 and described in a January…
ELMo (Embeddings from Language Models)
ELMo (Embeddings from Language Models) is a deep contextualized word embedding method, introduced in 2018 by the Allen Institute for AI (AI2) and the University of Washington
Embedding Layer
An embedding layer is a neural network component that acts as a trainable lookup table, mapping discrete integer indices (such as word IDs, user IDs, or category codes) to dense, continuous-valued vectors.
Embedding vector
An embedding vector is a dense, fixed-length array of real numbers that represents a discrete object (such as a word, sentence, image, audio clip, or graph node) as a point in a continuous vector space.
Embeddings
In machine learning, an embedding is a learned representation that maps an input into a vector, usually so that a downstream model or comparison rule can use relationships encoded in the vector's geometry.
Emily M. Bender
Emily M. Bender is an American linguist and a professor in the Department of Linguistics at the University of Washington, where she directs the Computational Linguistics Laboratory.
FLORES-200
FLORES-200 is a multilingual evaluation benchmark for machine translation systems, covering 200 languages across a wide range of language families, scripts, and resource levels.
Falcon (language model)
Falcon is a family of open-source large language models built by the Technology Innovation Institute (TII)
Fill-Mask Models
Fill-mask models are language models trained with a masked language modeling (MLM) objective, in which a fraction of the tokens in an input sequence are hidden behind a special [MASK] symbol and the model…
Fill-in-the-middle
Fill-in-the-middle (FIM) is a training objective and inference technique that lets an autoregressive language model generate text for a gap in the middle of a document, conditioned on both the text before the…
FineWeb
FineWeb is a large-scale, open pretraining dataset for large language models (LLMs) created by Hugging Face.
Full Softmax
Full softmax (also called the standard softmax or exact softmax) is the softmax computation that calculates a probability for every possible output class in a classification model
GLUE benchmark
The General Language Understanding Evaluation (GLUE) benchmark is a collection of nine natural language understanding (NLU) tasks designed to evaluate and compare the performance of language models across a…
GPT-1
GPT-1 is the first model in the GPT (Generative Pre-trained Transformer) series, a 117-million-parameter, 12-layer decoder-only Transformer released by OpenAI on June 11, 2018 in the paper "Improving Language…
GPT-2
GPT-2 is a family of autoregressive language models introduced by OpenAI on February 14, 2019.
GPT-3
GPT-3 (Generative Pre-trained Transformer 3) is a family of decoder-only, autoregressive large language models developed by OpenAI.
GPT-4
GPT-4 (Generative Pre-trained Transformer 4) is a large language model developed by OpenAI and released on March 14, 2023.
Gemini (language model)
Gemini is a family of natively multimodal large language models developed by Google DeepMind, first announced on December 6, 2023, that can reason across text, images, audio, video, and code within a single…
GloVe (Global Vectors for Word Representation)
GloVe (Global Vectors for Word Representation) is an unsupervised learning algorithm that turns words into dense numeric vectors by factorizing a word-word co-occurrence matrix built from a large text corpus.
Global-MMLU
Global-MMLU is a multilingual evaluation benchmark that extends the MMLU question-answering dataset across 42 languages, with designated subsets labeled culturally sensitive (CS) and culturally agnostic (CA).
Greedy decoding
Greedy decoding (also called greedy search or argmax decoding) is the simplest text-generation strategy used by autoregressive language models: at every step it picks the single highest-probability next token…
Grounding (artificial intelligence)
Grounding in artificial intelligence is the process of anchoring an AI system's outputs to verifiable
Hallucination
Hallucination in generative AI is the production of content that is unsupported, contradicted by an applicable source, factually wrong, internally inconsistent, or otherwise presented without an adequate basis.
Hanna Hajishirzi
Hanna Hajishirzi, whose scholarly publications generally use the name Hannaneh Hajishirzi, is a computer scientist whose research covers natural language processing, machine learning, generative AI, question…
Haystack (framework)
Haystack is an open-source AI orchestration framework developed by deepset, a Berlin-based company, for building production-ready natural language processing (NLP), retrieval-augmented generation (RAG), and AI…
HellaSwag
HellaSwag is a commonsense reasoning benchmark for language models, introduced by Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi in the 2019 paper "HellaSwag: Can a Machine Really…
HotpotQA
HotpotQA is a large-scale, multi-hop question answering dataset of about 112,779 crowd-authored question-and-answer pairs over English Wikipedia, whose answers cannot be found in any single paragraph and…
How to Pressure LLMs for Better Output
Pressuring large language models (LLMs) is a family of prompt engineering techniques that try to push a model toward better output by adding emotional weight, urgency, stakes, or coercion to the prompt.
HyDE (Hypothetical Document Embeddings)
HyDE (Hypothetical Document Embeddings) is a zero-shot dense retrieval technique that, instead of searching with the user's query, first prompts an instruction-following large language model to write a…
IFEval
IFEval (Instruction-Following Evaluation) is a benchmark of 541 prompts that measures how reliably large language models obey explicit, machine-checkable instructions such as "write in more than 400 words,"…
In-Context Learning
In-context learning (ICL) is the ability of a large language model to learn a new task at inference time by conditioning on a prompt that contains a few input-output examples (demonstrations)
InfiniteBench
InfiniteBench (stylized as ∞Bench) is a long-context benchmark that tests whether large language models (LLMs) can genuinely process and reason over inputs longer than 100,000 tokens, using 12 tasks that span…
Information Retrieval
Information retrieval (IR) is the study and engineering of systems that identify material likely to satisfy an information need.
Instruction Tuning
Instruction tuning is the post-pretraining training stage in which a large language model (LLM) is fine-tuned on a curated collection of (instruction
Jacob Devlin
Jacob Devlin is an American research scientist in natural language processing and machine learning, best known as the first author of BERT, the bidirectional language representation model that Google…
Jina Embeddings v3
Jina Embeddings v3 is a multilingual text embedding model released by Jina AI on September 18, 2024, with 570 million parameters, support for 89 languages, an 8,192 token context window, and a stack of…
Knowledge Editing
Knowledge editing (also called model editing) is a family of techniques for updating or correcting specific factual associations stored in the weights of a trained large language model without full retraining…
LAMBADA
LAMBADA (LAnguage Modeling Broadened to Account for Discourse Aspects) is a benchmark dataset designed to evaluate the ability of computational language models to understand broad discourse context.
LLaMA
LLaMA is the original name of a family of large language models developed by Meta AI.
Language Model
A language model is a model of patterns in language that assigns probabilities or comparable scores to linguistic sequences. Its units may be characters, words, subwords, bytes, or other tokens.
Large Concept Model
A Large Concept Model (LCM) is a research approach to language modeling, introduced by Meta AI's Fundamental AI Research (FAIR) group in December 2024
Large Language Model
A large language model (LLM) is an artificial intelligence system built on a transformer neural network with billions to trillions of parameters, trained on massive text corpora to predict the next token in a…
Latent Dirichlet allocation
Latent Dirichlet allocation (LDA) is a generative probabilistic model that discovers the hidden thematic structure in a collection of documents by treating each document as a mixture of a small number of…
Latent semantic analysis (LSA)
Latent semantic analysis (LSA), called latent semantic indexing (LSI) in information retrieval contexts, is an unsupervised technique that maps both terms and documents into a shared low-dimensional vector…
LegalBench
LegalBench is a collaboratively constructed benchmark for measuring legal reasoning in large language models (LLMs)
LibriSpeech
LibriSpeech is a freely available corpus of approximately 1,000 hours of 16 kHz read English speech that serves as the standard benchmark for training and evaluating automatic speech recognition (ASR) systems.
LiveBench
LiveBench is a contamination-limited benchmark for large language models (LLMs) that draws frequently updated questions from recent sources (math competitions, arXiv papers, news articles, and datasets) and…
Llama 3
Llama 3 is a family of open-weight large language models developed by Meta. Meta released the original Llama 3 checkpoints on April 18, 2024, in 8-billion-parameter and 70-billion-parameter sizes.
LlamaIndex
LlamaIndex is an open-source data framework for building large language model (LLM) applications, with a particular focus on retrieval-augmented generation (RAG) and document processing.
LoRA (Low-Rank Adaptation)
Low-Rank Adaptation, usually abbreviated LoRA, is a parameter-efficient fine-tuning method for adapting a pre-trained model.
LongBench
LongBench is a benchmark suite for evaluating the long-context understanding capabilities of large language models (LLMs).
Longformer
Longformer is a transformer architecture for processing long documents, introduced by Iz Beltagy, Matthew E. Peters
Low-Resource Languages
Low-resource languages, in natural language processing (NLP), are languages with little of the raw material that modern language technology depends on: digitized text, parallel translation corpora, annotated…
METEOR (metric)
METEOR (Metric for Evaluation of Translation with Explicit ORdering) is an automatic evaluation metric for machine translation and other text-generation tasks that scores a candidate sentence against one or…
MGSM (Multilingual Grade School Math)
MGSM (Multilingual Grade School Math) is a multilingual benchmark that measures the mathematical reasoning ability of large language models by taking 250 grade-school math word problems from GSM8K and…
MMTEB
MMTEB (Massive Multilingual Text Embedding Benchmark) is a large, community-built suite for evaluating text embedding models across more than 500 quality-controlled tasks and over 250 languages
MT-Bench
MT-Bench (Multi-Turn Benchmark) is a benchmark of 80 hand-written, two-turn questions that evaluates large language models (LLMs) on multi-turn conversation and instruction following by using a strong model…