Thursday, July 23, 2026
- Show-ov2Show-o is a unified multimodal model, introduced in 2024, that handles both multimodal understanding and visual generation inside a single Transformer.
- Transfusionv2Transfusion is a recipe, introduced by Meta in 2024, for training a single Transformer over a mixture of discrete text and continuous image data using two training objectives simultaneously: a…
- MaskGITv2MaskGIT, short for Masked Generative Image Transformer, is an image-synthesis method introduced by Google Research in the 2022 paper "MaskGIT: Masked Generative Image Transformer" by Huiwen Chang, Han Zhang…
- Masked Autoregressive (MAR) generationv2Masked Autoregressive (MAR) generation is an image-generation method introduced in the 2024 paper "Autoregressive Image Generation without Vector Quantization" by Tianhong Li, Yonglong Tian, He Li, Mingyang…
- Visual Autoregressive modeling (VAR)v3Visual Autoregressive modeling (VAR) is an image generation paradigm, introduced in 2024, that reframes autoregressive image synthesis as coarse-to-fine "next-scale prediction" rather than the conventional…
- DPM-Solverv3DPM-Solver is a fast, training-free high-order numerical solver for the ordinary differential equations (ODEs) that arise when sampling from diffusion models, and it lets a pretrained model generate…
- VQGAN (Taming Transformers)v3VQGAN (Vector Quantized Generative Adversarial Network) is a two-stage image-synthesis method that first compresses an image into a small grid of discrete codebook tokens with an adversarially trained…
- VQ-VAE (Vector Quantized Variational Autoencoder)v4VQ-VAE (Vector Quantized Variational Autoencoder) is a generative neural network that compresses data into a grid or sequence of discrete tokens drawn from a learned codebook
- Classifier-Free Guidance (CFG)v3Classifier-Free Guidance (CFG) is an inference-time technique that steers conditional generative models, especially diffusion models, by combining a single model's conditional and unconditional predictions and…
- Emergent misalignmentv3Emergent misalignment is an AI safety finding, first reported in February 2025, in which fine-tuning a large language model on a single narrow bad behavior causes it to become broadly misaligned across many…
- Causal scrubbingv3Causal scrubbing is a methodology in mechanistic interpretability for rigorously and quantitatively testing hypotheses about the internal computational structure of a neural network.
- Circuit Breakers (Representation Rerouting)v2Circuit Breakers are an AI safety method, introduced in 2024, that aims to make a large language model (LLM) or multimodal model robust to harmful generations by intervening directly on the model's internal…
- Dictionary learning (for interpretability)v3Dictionary learning, in the context of mechanistic interpretability, is the framework of decomposing the dense internal activations of a neural network into a sparse, weighted combination drawn from a large
- Gemma Scopev2Gemma Scope is an open, comprehensive suite of sparse autoencoders (SAEs) released by Google DeepMind in 2024 to support mechanistic interpretability research on its open-weight Gemma 2 language models.
- Gated SAEv3A Gated sparse autoencoder (Gated SAE) is a sparse-autoencoder architecture for mechanistic interpretability that splits the encoder into a gating path, which decides which features are active, and a magnitude…
- TopK SAEv3A TopK SAE (TopK sparse autoencoder) is a variant of sparse autoencoder that enforces sparsity by keeping only the K largest latent pre-activations for each input and zeroing all the rest
- JumpReLU SAEv4A JumpReLU sparse autoencoder (JumpReLU SAE) is a variant of the sparse autoencoder used in mechanistic interpretability whose encoder applies a learnable per-feature threshold that forces a feature to exactly…
- Position Interpolation (PI)v2Position Interpolation (PI) is a method for extending the context window of a pretrained large language model that uses rotary position embedding (RoPE).
- Self-Extendv2Self-Extend (written SelfExtend in the original paper) is a training-free technique that lets a pretrained large language model process inputs much longer than the context window it was trained on, with no…
- DeepNorm / DeepNetv3DeepNorm is a normalization and weight initialization scheme for Transformer networks that makes the training of very deep models stable.
- Register tokens (Vision Transformers Need Registers)v2Register tokens are a small set of extra learnable tokens added to the input sequence of a Vision Transformer (ViT) so the network has a dedicated place to carry out internal, image-level computation.
- Attention sinkv3An attention sink is an empirical phenomenon in Transformer language models in which a large fraction of each attention head's weight concentrates on a few tokens at the very start of the sequence, most often…
- PEER (Parameter Efficient Expert Retrieval / Mixture of a Million Experts)v3PEER, short for Parameter Efficient Expert Retrieval, is a neural network layer for Transformer models that replaces the dense feed-forward block with a sparse mixture of experts drawn from an extremely large…
- Expert Choice routingv2Expert Choice routing (often abbreviated EC) is a routing method for mixture of experts (MoE) layers in neural networks, introduced by researchers at Google in 2022.
- Soft MoEv2Soft MoE (Soft Mixture of Experts) is a fully differentiable variant of the sparse mixture of experts (MoE) layer.
- Gated DeltaNetv3Gated DeltaNet is a linear attention sequence model that augments the delta rule, a key-value error-correction update used in the earlier DeltaNet architecture
- H-Net (dynamic chunking)v2H-Net, short for Hierarchical Network, is a tokenizer-free neural sequence model that learns to segment raw bytes into content-adaptive "chunks" as part of ordinary end-to-end training, rather than relying on…
- Mixture-of-Recursions (MoR)v2Mixture-of-Recursions (MoR) is a Transformer architecture, introduced in 2025, that unifies two previously separate strategies for building efficient language models: parameter sharing through recursion, and…
- Hymbav2Hymba is a hybrid-head neural-network architecture for small language models introduced by NVIDIA researchers in November 2024.
- Lightning Attentionv4Lightning Attention is an IO-aware (input/output aware) implementation of linear attention that lets the method reach its theoretical linear-time complexity in practice
- Mixture of Block Attention (MoBA)v2Mixture of Block Attention (MoBA) is a trainable block-sparse attention mechanism introduced in February 2025 by researchers at Moonshot AI, the company that builds the Kimi family of large language models
- Native Sparse Attention (NSA)v3Native Sparse Attention (NSA) is a hardware-aligned, natively trainable sparse attention mechanism introduced in February 2025 by DeepSeek, in collaboration with researchers at Peking University and the…
- Quantization-Aware Training (QAT)v3Quantization-aware training (QAT) is a model compression technique in which the effects of quantization are simulated during the training or fine-tuning of a neural network, so that the model learns parameter…
- Contrastive decodingv3Contrastive decoding (CD) is a decoding strategy for text generation from a large language model that selects tokens by contrasting two models of different sizes.
- Self-speculative decoding (LayerSkip)v2Self-speculative decoding is a family of speculative decoding methods that accelerate large language model (LLM) inference by using the target model itself, run in a cheaper reduced-depth mode
- Prompt lookup decodingv2Prompt lookup decoding (PLD), also called n-gram speculative decoding, is an inference acceleration method for large language models that speeds up text generation without changing the model or its outputs.
- Prefix caching (automatic prefix caching)v2Prefix caching is an inference optimization for large language model serving that stores and reuses the key-value (KV) cache computed for a shared prompt prefix, so that the prefill for that prefix is computed…
- Chunked prefillv3Chunked prefill is a scheduling technique for large language model serving that splits the processing of a long input prompt (the prefill) into smaller, fixed-size token chunks and combines each chunk with the…
- KV-cache quantizationv2KV cache quantization is a family of large language model inference optimizations that store the attention key and value (KV) cache in low-bit numeric formats, typically 2 to 4 bits per value
- AQLM (Additive Quantization of Language Models)v2AQLM, short for Additive Quantization of Language Models, is a weight-only post-training quantization method that compresses the weights of a large language model to roughly 2 to 3 bits per parameter while…
- SpinQuantv2SpinQuant is a post-training quantization method for large language models that inserts learned rotation matrices into a transformer network to make its weights, activations, and KV cache easier to represent…
- QuIP / QuIP#v3QuIP (Quantization with Incoherence Processing) is a family of weight-only post-training quantization methods for large language models developed in the RelaxML group at Cornell University
- Depth up-scaling (DUS)v3Depth up-scaling (DUS) is a model-scaling method that builds a deeper large language model by duplicating and stacking the layers of an existing pretrained Transformer checkpoint and then continuing to…
- Sparse upcyclingv2Sparse upcycling is a technique for building a sparsely activated mixture of experts (MoE) model by initializing it from an already trained dense Transformer checkpoint
- Selective Language Modeling (Rho-1)v2Selective Language Modeling (SLM) is a pretraining objective for language models that applies the training loss to only a chosen subset of tokens rather than to every token in the corpus.
- DoReMiv2DoReMi (Domain Reweighting with Minimax Optimization) is a method for automatically choosing the proportions, or "domain weights," of each data source in a pretraining corpus for a large language model.
- Instruction backtranslation (Humpback)v2Instruction backtranslation is a self-alignment method for generating instruction tuning data, introduced by researchers at Meta AI in the paper "Self-Alignment with Instruction Backtranslation," first posted…
- Evol-Instructv3Evol-Instruct is a method for automatically generating large instruction tuning datasets by prompting a large language model to rewrite
- Model soupsv3Model soups is a weight-averaging technique (a form of model merging) that combines several independently fine-tuned neural networks into a single model by averaging their parameters.
- Task arithmeticv4Task arithmetic is a model-editing technique that steers the behavior of a neural network by adding or subtracting vectors in its weight space.
- DARE (Drop And REscale)v2DARE (Drop And REscale) is a training-free preprocessing technique for model merging that sparsifies the parameter changes introduced by fine-tuning before those changes are combined across several models.
- TIES-Mergingv4TIES-Merging is a training-free model merging method that combines several models fine-tuned from a shared pre-trained checkpoint into one multitask model while explicitly resolving the interference between…
- Best-of-N samplingv3Best-of-N sampling (BoN) is an inference-time method that improves a large language model output by drawing N independent candidate responses to the same prompt, scoring each with a reward model, verifier, or…
- ReST / ReST-EM (Reinforced Self-Training)v2ReST (Reinforced Self-Training) is a family of self-training algorithms for large language models that improve a model by fine-tuning it on its own filtered outputs instead of on additional human-written data.
- Quiet-STaRv2Quiet-STaR is a self-supervised training method that teaches a large language model to generate short, token-level internal "thoughts," or rationales, that help it predict the text that follows
- STaR (Self-Taught Reasoner)v3STaR (Self-Taught Reasoner) is a self-training method that teaches a large language model to reason by having it generate its own chain-of-thought rationales, keeping only the rationales that lead to correct…
- VAPO (Value-based Augmented PPO)v2VAPO (Value-based Augmented Proximal Policy Optimization) is a reinforcement learning framework for training large language models on long chain-of-thought reasoning tasks.
- DAPO (Decoupled Clip and Dynamic Sampling Policy Optimization)v2DAPO, short for Decoupled Clip and Dynamic sAmpling Policy Optimization, is an open-source reinforcement learning algorithm and training system for large language models, introduced in March 2025 by…
- Group Sequence Policy Optimization (GSPO)v3Group Sequence Policy Optimization (GSPO) is a reinforcement learning algorithm for training large language models, introduced by the Qwen team at Alibaba in July 2025 .
- Wang Jianv2Wang Jian (Chinese: 王坚; born October 1962) is a Chinese computer scientist and technology executive who founded Alibaba Cloud, the cloud computing division of Alibaba Group, and served as the chief architect…
- Liu Zhiyuanv2Liu Zhiyuan (Chinese: 刘知远) is a Chinese computer scientist and entrepreneur known for his research in natural language processing, knowledge graphs, and representation learning, and for his work on efficient…
- Kai Yu (Yu Kai)v2Kai Yu (Chinese: 余凯; pinyin: Yu Kai) is a Chinese computer scientist and entrepreneur who is the founder, chairman, and chief executive officer of Horizon Robotics, a Beijing company that designs…
- Yin Qi (Qi Yin)v2Yin Qi (Chinese: 印奇; born January 1988), sometimes written in English as Qi Yin, is a Chinese computer scientist and technology entrepreneur best known as a co-founder and longtime chief executive of Megvii…
- Wang Xiaochuanv3Wang Xiaochuan (born October 3, 1978) is a Chinese computer scientist and entrepreneur, the founder and chief executive of Baichuan (Baichuan Intelligence)
- Zhang Peng (Zhipu AI)v4Zhang Peng (Chinese: 张鹏) is a Chinese computer scientist and technology executive who serves as co-founder and chief executive officer of Zhipu AI
- Liang Rubov2Liang Rubo (Chinese: 梁汝波; born 1983) is a Chinese technology executive who is the co-founder, chairman, and chief executive officer of ByteDance, the Beijing-based company behind the short-video apps Douyin…
- Robin Liv4Robin Li (Li Yanhong; born November 17, 1968) is a Chinese software engineer and entrepreneur who co-founded Baidu in January 2000 and has led it ever since as chairman and chief executive officer .
- Jacob Steinhardtv2Jacob Steinhardt is an American statistician and computer scientist who is an associate professor of statistics, with a joint appointment in electrical engineering and computer sciences (EECS), at the…
- Christopher Rév3Christopher Re (his surname is sometimes written with an acute accent over the "e") is an American computer scientist and a professor of computer science at Stanford University
- Sepp Hochreiterv2Sepp Hochreiter (born February 14, 1967) is a German computer scientist regarded as one of the founders of modern deep learning.
- Judea Pearlv2Judea Pearl (born September 4, 1936) is an Israeli-American computer scientist, electrical engineer, and philosopher who is one of the principal architects of the modern approach to reasoning under uncertainty…
- Anton Korinekv4Anton Korinek is an Austrian-American economist and a professor in the Department of Economics and the Darden School of Business at the University of Virginia (UVA), where he founded and directs the Economics…
- Erik Brynjolfssonv2Erik Brynjolfsson is an American economist who is among the most influential scholars studying the economics of information technology and artificial intelligence.
- J. Zico Kolterv2J. Zico Kolter (full name Jeremy Zico Kolter) is an American computer scientist who is a professor and head of the Machine Learning Department at Carnegie Mellon University.
- Roman Yampolskiyv2Roman Vladimirovich Yampolskiy (born August 13, 1979) is a Latvian-American computer scientist and a tenured associate professor in the Department of Computer Engineering and Computer Science at the University…
- Audrey Tangv3Audrey Tang (Chinese: 唐鳳; born Tang Tsung-han, April 18, 1981) is a Taiwanese free-software programmer, civic technologist, and politician who served as Taiwan's first Minister of Digital Affairs from 2022 to…
- Tristan Harrisv2Tristan Harris is an American technology ethicist, public advocate, and entrepreneur best known as a co-founder and the president of the Center for Humane Technology
- Kate Crawfordv3Kate Crawford (born 1974) is an Australian scholar, author, and artist known as one of the leading critical voices on the politics of artificial intelligence.
- Gillian Hadfieldv2Gillian Kereldena Hadfield (born July 14, 1961) is a Canadian-American legal scholar and economist whose research sits at the intersection of law, economics, and artificial intelligence.
- Connor Leahyv2Connor Leahy is a German-American artificial intelligence researcher and entrepreneur known for his work on open-source large language models and for his advocacy on the risks that advanced AI poses to…
- Ian Hogarthv2Ian Hogarth is a British technology entrepreneur, investor, and writer who serves as the founding chair of the UK AI Safety Institute
- Geoffrey Irvingv3Geoffrey Irving is a computer scientist and artificial intelligence safety researcher who, until 2026, served as chief scientist of the UK AI Security Institute (AISI), the British government body that…
- Marius Hobbhahnv3Marius Hobbhahn is a German AI safety researcher and the co-founder and chief executive officer of Apollo Research, a London-based organization that studies and evaluates deceptive behavior, often called…
- Emmett Shearv3Emmett Shear (born 1983) is an American entrepreneur, software engineer, and investor best known as a co-founder and longtime chief executive of the live-streaming platform Twitch.
- Daniel Kokotajlov2Daniel Kokotajlo is a researcher and forecaster of artificial intelligence who serves as executive director of the AI Futures Project, a nonprofit that studies the trajectory and societal impact of advanced AI.
- Elizabeth "Beth" Barnesv2Elizabeth "Beth" Barnes is a British AI safety researcher and the founder and chief executive of METR (Model Evaluation and Threat Research)
- Edwin Chenv3Edwin Chen is an American entrepreneur and machine learning engineer who is the founder and chief executive officer of Surge AI, a company that supplies human-generated training data (data labeling)…
- David Luanv2David Luan is an American artificial intelligence executive and entrepreneur best known as the co-founder and chief executive of Adept AI, a startup that helped pioneer AI agents capable of operating software…
- Daphne Kollerv2Daphne Koller (born August 27, 1968) is an Israeli-American computer scientist and entrepreneur who is among the most influential figures in applying machine learning to both education and the life sciences.
- Jonathan Franklev2Jonathan Frankle is an American computer scientist who serves as Chief AI Scientist at Databricks, where he leads the company's Mosaic Research team.
- Naveen Raov2Naveen Rao is an American entrepreneur, computer architect, and neuroscientist who has founded three artificial intelligence companies and twice sold a startup to a large technology firm.
- Ali Ghodsiv3Ali Ghodsi (born 1978) is the co-founder and chief executive officer of Databricks, one of the most valuable private companies in the world and a leading vendor of data and artificial intelligence software.
- Amjad Masadv2Amjad Masad (born 1987 or 1988) is a Jordanian American software engineer and entrepreneur who is the co-founder and chief executive officer of Replit, a browser based software development platform.
- Demi Guov2Demi Guo (born around 1998) is a Chinese-born American technology entrepreneur and former artificial intelligence researcher who is the co-founder and chief executive officer of Pika
- Cristóbal Valenzuelav2Cristobal Valenzuela (born 1989) is a Chilean technologist, artist, and entrepreneur who co-founded Runway, the New York based generative AI company, in 2018 and has served as its chief executive officer since…
- Douwe Kielav2Douwe Kiela (born 1986) is a Dutch-American computer scientist and entrepreneur best known for his work on retrieval-augmented generation (RAG) and on multimodal natural language processing.
- Stanislas Poluv2Stanislas Polu is a French software engineer, AI researcher, and entrepreneur. He is the co-founder and chief technology officer of Dust, a Paris-based company that builds an enterprise platform for AI agents.
- Eric Steinbergerv3Eric Steinberger is an Austrian computer scientist and technology entrepreneur who is the co-founder and chief executive officer of Magic.dev
- Eiso Kantv2Eiso Kant is a Dutch software entrepreneur and technology executive who is a co-founder and the chief technology officer of Poolside, a company building foundation models for software development.
- Varun Mohanv2Varun Mohan is an American software engineer and entrepreneur who co-founded the artificial intelligence coding company Codeium, later renamed Windsurf, and served as its chief executive from 2021 until 2025.
