Monday, June 8, 2026
- TheAgentCompanyv2TheAgentCompany is an AI benchmark that evaluates AI agents on long-horizon, economically valuable knowledge work inside a self-hosted simulation of a small software company.
- LAB-Benchv2LAB-Bench (the Language Agent Biology Benchmark) is an AI benchmark of more than 2,400 multiple-choice questions built to measure how well language models and AI agents can perform practical biology research…
- ChemBenchv2ChemBench is an automated AI benchmark that measures the chemical knowledge, reasoning, and safety judgment of large language models and compares their performance against expert human chemists.
- KernelBenchv4KernelBench is an AI benchmark and open-source evaluation environment that measures how well large language models can write fast and correct GPU kernels.
- ScienceAgentBenchv2ScienceAgentBench is an AI benchmark for evaluating whether language model agents can perform real, data-driven scientific analysis by writing and executing code.
- PutnamBenchv2PutnamBench is a benchmark for evaluating automated theorem proving systems, especially neural and large-language-model-based provers, on competition mathematics.
- Omni-MATHv2Omni-MATH is an AI benchmark of Olympiad-level competition mathematics, introduced in October 2024 to measure the mathematical reasoning ability of large language models on problems far harder than those in…
- OlympiadBenchv2OlympiadBench is an AI benchmark of 8,476 Olympiad-level mathematics and physics problems, designed to test the advanced scientific reasoning of large language models and large multimodal models.
- MMLU-Reduxv2MMLU-Redux is a manually re-annotated, error-corrected subset of the Massive Multitask Language Understanding (MMLU) benchmark
- AGIEvalv2AGIEval is an AI benchmark for evaluating foundation models on tasks that were originally designed for, and taken by, humans.
- FormulaOnev3FormulaOne is an AI benchmark for evaluating whether a model can design and implement dynamic programming algorithms for graph problems.
- EnigmaEvalv3EnigmaEval is an AI benchmark of long, complex multimodal puzzles drawn from real-world puzzle hunts, designed to measure the unstructured, creative, multi-step reasoning abilities of frontier AI models.
- ARC-AGI-2v4ARC-AGI-2 (Abstraction and Reasoning Corpus for Artificial General Intelligence 2) is an abstract reasoning benchmark for artificial intelligence, released on March 24, 2025 by the ARC Prize Foundation
- MoE load-balancing lossv3The MoE load balancing loss is an auxiliary training objective used in sparse mixture of experts (MoE) neural networks to keep work spread evenly across the expert sub-networks.
- DualPipev3DualPipe is a bidirectional pipeline parallelism scheduling algorithm created by DeepSeek-AI to almost fully overlap computation with communication and shrink the idle time, called the pipeline "bubble," that…
- NLWebv2NLWeb (short for Natural Language Web) is an open project from Microsoft that makes it easy to add a natural-language conversational interface to a website, drawing on the site's own existing structured data…
- RAPTORv2RAPTOR (Recursive Abstractive Processing for Tree-Organized Retrieval) is a retrieval method for retrieval-augmented generation introduced in a 2024 paper by researchers at Stanford University.
- Corrective RAG (CRAG)v2Corrective Retrieval Augmented Generation (CRAG) is a method for improving the robustness of retrieval-augmented generation (RAG) when the underlying retrieval step returns irrelevant, incomplete, or factually…
- Self-RAGv2Self-RAG (Self-Reflective Retrieval-Augmented Generation) is a framework that trains a single large language model to adaptively decide when to retrieve external passages, to generate text grounded in those…
- MemGPTv4MemGPT (short for Memory-GPT) is a system and agent design pattern that gives large language model agents long-term memory by managing data between the model's bounded context window and external storage
- Sleep-time computev2Sleep-time compute is a technique for large language model inference in which a model uses idle periods, before any user query has arrived, to "think" about a known context offline and pre-compute a richer…
- Coconut (Chain of Continuous Thought)v3Coconut (Chain of Continuous Thought) is a reasoning paradigm for large language models introduced by researchers at FAIR at Meta, Meta's Fundamental AI Research lab, and the University of California
- SDEditv2SDEdit (Stochastic Differential Editing) is a method for guided image synthesis and editing that turns a rough user guide, such as a stroke painting, a coarse collage, or a real photograph with edits pasted…
- Prompt-to-Promptv2Prompt-to-Prompt is a training-free image editing technique for text-conditioned diffusion models that edits a generated image by manipulating the model's cross-attention maps when the text prompt is changed.
- Textual Inversionv3Textual Inversion is a technique for personalizing text-to-image diffusion models that teaches a frozen model a new visual concept from only three to five example images by learning a single new "pseudo-word"…
- DreamBoothv3DreamBooth is a subject-driven fine-tuning method for text-to-image diffusion models that personalizes a pretrained model to a specific subject, for example a particular dog, toy, or person, from just 3 to 5…
- IP-Adapterv2IP-Adapter (short for Image Prompt Adapter) is a lightweight neural network module that adds image-prompt conditioning to a pretrained text-to-image diffusion model, allowing a reference image to guide…
- Diffusion Forcingv2Diffusion Forcing is a training paradigm for sequence generative modeling introduced in 2024 that assigns each token in a sequence its own independent, randomly sampled noise level during training.
- Show-ov2Show-o is a unified multimodal model, introduced in 2024, that handles both multimodal understanding and visual generation inside a single Transformer.
- Transfusionv2Transfusion is a recipe, introduced by Meta in 2024, for training a single Transformer over a mixture of discrete text and continuous image data using two training objectives simultaneously: a…
- MaskGITv4MaskGIT, short for Masked Generative Image Transformer, is an image-synthesis method introduced by Google Research in the 2022 paper "MaskGIT: Masked Generative Image Transformer" by Huiwen Chang, Han Zhang…
- Masked Autoregressive (MAR) generationv2Masked Autoregressive (MAR) generation is an image-generation method introduced in the 2024 paper "Autoregressive Image Generation without Vector Quantization" by Tianhong Li, Yonglong Tian, He Li, Mingyang…
- Visual Autoregressive modeling (VAR)v3Visual Autoregressive modeling (VAR) is an image generation paradigm, introduced in 2024, that reframes autoregressive image synthesis as coarse-to-fine "next-scale prediction" rather than the conventional…
- DPM-Solverv3DPM-Solver is a fast, training-free high-order numerical solver for the ordinary differential equations (ODEs) that arise when sampling from diffusion models, and it lets a pretrained model generate…
- VQGAN (Taming Transformers)v3VQGAN (Vector Quantized Generative Adversarial Network) is a two-stage image-synthesis method that first compresses an image into a small grid of discrete codebook tokens with an adversarially trained…
- VQ-VAE (Vector Quantized Variational Autoencoder)v4VQ-VAE (Vector Quantized Variational Autoencoder) is a generative neural network that compresses data into a grid or sequence of discrete tokens drawn from a learned codebook
- Classifier-Free Guidance (CFG)v3Classifier-Free Guidance (CFG) is an inference-time technique that steers conditional generative models, especially diffusion models, by combining a single model's conditional and unconditional predictions and…
- Emergent misalignmentv3Emergent misalignment is an AI safety finding, first reported in February 2025, in which fine-tuning a large language model on a single narrow bad behavior causes it to become broadly misaligned across many…
- Causal scrubbingv3Causal scrubbing is a methodology in mechanistic interpretability for rigorously and quantitatively testing hypotheses about the internal computational structure of a neural network.
- Circuit Breakers (Representation Rerouting)v2Circuit Breakers are an AI safety method, introduced in 2024, that aims to make a large language model (LLM) or multimodal model robust to harmful generations by intervening directly on the model's internal…
- Dictionary learning (for interpretability)v3Dictionary learning, in the context of mechanistic interpretability, is the framework of decomposing the dense internal activations of a neural network into a sparse, weighted combination drawn from a large
- Gemma Scopev2Gemma Scope is an open, comprehensive suite of sparse autoencoders (SAEs) released by Google DeepMind in 2024 to support mechanistic interpretability research on its open-weight Gemma 2 language models.
- Gated SAEv3A Gated sparse autoencoder (Gated SAE) is a sparse-autoencoder architecture for mechanistic interpretability that splits the encoder into a gating path, which decides which features are active, and a magnitude…
- TopK SAEv3A TopK SAE (TopK sparse autoencoder) is a variant of sparse autoencoder that enforces sparsity by keeping only the K largest latent pre-activations for each input and zeroing all the rest
- JumpReLU SAEv4A JumpReLU sparse autoencoder (JumpReLU SAE) is a variant of the sparse autoencoder used in mechanistic interpretability whose encoder applies a learnable per-feature threshold that forces a feature to exactly…
- Position Interpolation (PI)v2Position Interpolation (PI) is a method for extending the context window of a pretrained large language model that uses rotary position embedding (RoPE).
- Self-Extendv2Self-Extend (written SelfExtend in the original paper) is a training-free technique that lets a pretrained large language model process inputs much longer than the context window it was trained on, with no…
- DeepNorm / DeepNetv3DeepNorm is a normalization and weight initialization scheme for Transformer networks that makes the training of very deep models stable.
- Register tokens (Vision Transformers Need Registers)v2Register tokens are a small set of extra learnable tokens added to the input sequence of a Vision Transformer (ViT) so the network has a dedicated place to carry out internal, image-level computation.
- Attention sinkv3An attention sink is an empirical phenomenon in Transformer language models in which a large fraction of each attention head's weight concentrates on a few tokens at the very start of the sequence, most often…
- PEER (Parameter Efficient Expert Retrieval / Mixture of a Million Experts)v3PEER, short for Parameter Efficient Expert Retrieval, is a neural network layer for Transformer models that replaces the dense feed-forward block with a sparse mixture of experts drawn from an extremely large…
- Expert Choice routingv2Expert Choice routing (often abbreviated EC) is a routing method for mixture of experts (MoE) layers in neural networks, introduced by researchers at Google in 2022.
- Soft MoEv2Soft MoE (Soft Mixture of Experts) is a fully differentiable variant of the sparse mixture of experts (MoE) layer.
- Gated DeltaNetv3Gated DeltaNet is a linear attention sequence model that augments the delta rule, a key-value error-correction update used in the earlier DeltaNet architecture
- H-Net (dynamic chunking)v2H-Net, short for Hierarchical Network, is a tokenizer-free neural sequence model that learns to segment raw bytes into content-adaptive "chunks" as part of ordinary end-to-end training, rather than relying on…
- Mixture-of-Recursions (MoR)v2Mixture-of-Recursions (MoR) is a Transformer architecture, introduced in 2025, that unifies two previously separate strategies for building efficient language models: parameter sharing through recursion, and…
- Hymbav2Hymba is a hybrid-head neural-network architecture for small language models introduced by NVIDIA researchers in November 2024.
- DeepSeek Sparse Attention (DSA)v5DeepSeek Sparse Attention (DSA) is a trainable, fine-grained sparse attention mechanism introduced by the Chinese AI company DeepSeek in its experimental model DeepSeek-V3.2-Exp, released on September 29, 2025.
- Lightning Attentionv4Lightning Attention is an IO-aware (input/output aware) implementation of linear attention that lets the method reach its theoretical linear-time complexity in practice
- Mixture of Block Attention (MoBA)v2Mixture of Block Attention (MoBA) is a trainable block-sparse attention mechanism introduced in February 2025 by researchers at Moonshot AI, the company that builds the Kimi family of large language models
- Native Sparse Attention (NSA)v3Native Sparse Attention (NSA) is a hardware-aligned, natively trainable sparse attention mechanism introduced in February 2025 by DeepSeek, in collaboration with researchers at Peking University and the…
- Quantization-Aware Training (QAT)v3Quantization-aware training (QAT) is a model compression technique in which the effects of quantization are simulated during the training or fine-tuning of a neural network, so that the model learns parameter…
- Contrastive decodingv3Contrastive decoding (CD) is a decoding strategy for text generation from a large language model that selects tokens by contrasting two models of different sizes.
- Self-speculative decoding (LayerSkip)v2Self-speculative decoding is a family of speculative decoding methods that accelerate large language model (LLM) inference by using the target model itself, run in a cheaper reduced-depth mode
- Prompt lookup decodingv2Prompt lookup decoding (PLD), also called n-gram speculative decoding, is an inference acceleration method for large language models that speeds up text generation without changing the model or its outputs.
- Prefix caching (automatic prefix caching)v2Prefix caching is an inference optimization for large language model serving that stores and reuses the key-value (KV) cache computed for a shared prompt prefix, so that the prefill for that prefix is computed…
- Chunked prefillv3Chunked prefill is a scheduling technique for large language model serving that splits the processing of a long input prompt (the prefill) into smaller, fixed-size token chunks and combines each chunk with the…
- KV-cache quantizationv2KV cache quantization is a family of large language model inference optimizations that store the attention key and value (KV) cache in low-bit numeric formats, typically 2 to 4 bits per value
- AQLM (Additive Quantization of Language Models)v2AQLM, short for Additive Quantization of Language Models, is a weight-only post-training quantization method that compresses the weights of a large language model to roughly 2 to 3 bits per parameter while…
- SpinQuantv2SpinQuant is a post-training quantization method for large language models that inserts learned rotation matrices into a transformer network to make its weights, activations, and KV cache easier to represent…
- QuIP / QuIP#v3QuIP (Quantization with Incoherence Processing) is a family of weight-only post-training quantization methods for large language models developed in the RelaxML group at Cornell University
- Depth up-scaling (DUS)v3Depth up-scaling (DUS) is a model-scaling method that builds a deeper large language model by duplicating and stacking the layers of an existing pretrained Transformer checkpoint and then continuing to…
- Sparse upcyclingv2Sparse upcycling is a technique for building a sparsely activated mixture of experts (MoE) model by initializing it from an already trained dense Transformer checkpoint
- Selective Language Modeling (Rho-1)v2Selective Language Modeling (SLM) is a pretraining objective for language models that applies the training loss to only a chosen subset of tokens rather than to every token in the corpus.
- DoReMiv2DoReMi (Domain Reweighting with Minimax Optimization) is a method for automatically choosing the proportions, or "domain weights," of each data source in a pretraining corpus for a large language model.
- Instruction backtranslation (Humpback)v2Instruction backtranslation is a self-alignment method for generating instruction tuning data, introduced by researchers at Meta AI in the paper "Self-Alignment with Instruction Backtranslation," first posted…
- Evol-Instructv3Evol-Instruct is a method for automatically generating large instruction tuning datasets by prompting a large language model to rewrite
- Model soupsv3Model soups is a weight-averaging technique (a form of model merging) that combines several independently fine-tuned neural networks into a single model by averaging their parameters.
- Task arithmeticv4Task arithmetic is a model-editing technique that steers the behavior of a neural network by adding or subtracting vectors in its weight space.
- DARE (Drop And REscale)v2DARE (Drop And REscale) is a training-free preprocessing technique for model merging that sparsifies the parameter changes introduced by fine-tuning before those changes are combined across several models.
- TIES-Mergingv4TIES-Merging is a training-free model merging method that combines several models fine-tuned from a shared pre-trained checkpoint into one multitask model while explicitly resolving the interference between…
- Best-of-N samplingv3Best-of-N sampling (BoN) is an inference-time method that improves a large language model output by drawing N independent candidate responses to the same prompt, scoring each with a reward model, verifier, or…
- ReST / ReST-EM (Reinforced Self-Training)v2ReST (Reinforced Self-Training) is a family of self-training algorithms for large language models that improve a model by fine-tuning it on its own filtered outputs instead of on additional human-written data.
- Quiet-STaRv2Quiet-STaR is a self-supervised training method that teaches a large language model to generate short, token-level internal "thoughts," or rationales, that help it predict the text that follows
- STaR (Self-Taught Reasoner)v3STaR (Self-Taught Reasoner) is a self-training method that teaches a large language model to reason by having it generate its own chain-of-thought rationales, keeping only the rationales that lead to correct…
- VAPO (Value-based Augmented PPO)v2VAPO (Value-based Augmented Proximal Policy Optimization) is a reinforcement learning framework for training large language models on long chain-of-thought reasoning tasks.
- DAPO (Decoupled Clip and Dynamic Sampling Policy Optimization)v2DAPO, short for Decoupled Clip and Dynamic sAmpling Policy Optimization, is an open-source reinforcement learning algorithm and training system for large language models, introduced in March 2025 by…
- Group Sequence Policy Optimization (GSPO)v3Group Sequence Policy Optimization (GSPO) is a reinforcement learning algorithm for training large language models, introduced by the Qwen team at Alibaba in July 2025.
- Wang Jianv2Wang Jian (Chinese: 王坚; born October 1962) is a Chinese computer scientist and technology executive who founded Alibaba Cloud, the cloud computing division of Alibaba Group, and served as the chief architect…
- Liu Zhiyuanv2Liu Zhiyuan (Chinese: 刘知远) is a Chinese computer scientist and entrepreneur known for his research in natural language processing, knowledge graphs, and representation learning, and for his work on efficient…
- Kai Yu (Yu Kai)v2Kai Yu (Chinese: 余凯; pinyin: Yu Kai) is a Chinese computer scientist and entrepreneur who is the founder, chairman, and chief executive officer of Horizon Robotics, a Beijing company that designs…
- Yin Qi (Qi Yin)v2Yin Qi (Chinese: 印奇; born January 1988), sometimes written in English as Qi Yin, is a Chinese computer scientist and technology entrepreneur best known as a co-founder and longtime chief executive of Megvii…
- Xu Liv3Xu Li (Chinese: 徐立; born in the 1980s) is a Chinese computer scientist and technology entrepreneur who co-founded SenseTime, one of China's largest artificial intelligence companies, and serves as its…
- Jiang Daxinv6Jiang Daxin (Chinese: 姜大昕) is a Chinese computer scientist and technology entrepreneur who is the founder and chief executive officer of StepFun (Chinese: 阶跃星辰), a Shanghai-based artificial intelligence…
- Wang Xiaochuanv3Wang Xiaochuan (born October 3, 1978) is a Chinese computer scientist and entrepreneur, the founder and chief executive of Baichuan (Baichuan Intelligence)
- Zhang Peng (Zhipu AI)v7Zhang Peng (Chinese: 张鹏) is a Chinese computer scientist and technology executive who serves as co-founder, executive director, chief executive officer and general manager of Zhipu AI
- Yan Junjiev8Yan Junjie (闫俊杰, born 1989) is a Chinese artificial intelligence researcher and entrepreneur who is the founder, chairman, chief executive officer, and chief technology officer of MiniMax, the Shanghai-based…
- Liang Rubov2Liang Rubo (Chinese: 梁汝波; born 1983) is a Chinese technology executive who is the co-founder, chairman, and chief executive officer of ByteDance, the Beijing-based company behind the short-video apps Douyin…
- Robin Liv4Robin Li (Li Yanhong; born November 17, 1968) is a Chinese software engineer and entrepreneur who co-founded Baidu in January 2000 and has led it ever since as chairman and chief executive officer.
- Jacob Steinhardtv2Jacob Steinhardt is an American statistician and computer scientist who is an associate professor of statistics, with a joint appointment in electrical engineering and computer sciences (EECS), at the…
