Papers

The most influential AI research papers are the small set of publications that introduced the architectures, training methods, and benchmarks that every modern system is built on, from "Attention Is All You Need" (the 2017 paper that introduced the transformer and underlies essentially every large language model) to AlexNet, GPT-3, Chinchilla, and DeepSeek-R1. This page is a chronological index of those papers. It covers the foundations of deep learning (LeNet, AlexNet, ImageNet, ResNet), the transformer era (Attention Is All You Need, BERT, GPT, T5), the scaling era (GPT-3, Chinchilla, PaLM, LLaMA), the alignment era (InstructGPT, RLHF, Constitutional AI, DPO), and the multimodal and reasoning era (GPT-4, Gemini, Llama 3, DeepSeek-R1). It also tracks landmark work in computer vision, reinforcement learning, audio, robotics, and scientific AI (AlphaGo, MuZero, AlphaFold).

What makes a paper important?

There is no committee that ranks AI papers. Importance is a rough mix of three things: technical novelty, citation count, and downstream impact on products people actually use. A paper that introduces a new architecture (the transformer, residual networks, mixture of experts) tends to stay important for years because every later paper builds on top of it. A paper that introduces a benchmark (ImageNet, GLUE, SuperGLUE, MMLU) shapes what the next decade of research optimizes for. A paper that opens up a new capability (GANs, diffusion, in-context learning, chain of thought) gets re-cited every time someone tries to extend or critique it.

There is also a softer kind of importance: papers that change how the field talks to itself. "Sparks of AGI" did this for GPT-4. "Emergent Abilities of Large Language Models" did it for scaling. "Constitutional AI" did it for alignment without exhaustive human labeling. These papers are not always the most technically deep, but they reframe debates that everyone else then has to respond to.

A few practical filters are useful when reading the table below:

• Did the paper introduce a model that other people kept training on (BERT, GPT-2, LLaMA, Mistral 7B)?
• Did it open up a new task or capability (Word2Vec for embeddings, CLIP for vision and language, Whisper for speech, AlphaFold for biology)?
• Did it become a standard reference cited by every follow-up paper (Adam, ResNet, Attention Is All You Need, Chinchilla, Scaling Laws)?
• Did it shift how labs build systems (RLHF, DPO, Constitutional AI, Chain of Thought)?

The table below skews toward papers that meet at least one of those tests. It is not exhaustive. The "Important Papers" section lists the canonical first appearances of major ideas. The "Other Papers" section lists notable follow-ups, benchmarks, and applied work.

How is the table organized?

Dates use the arXiv submission date (v1) when available, the conference or journal publication date otherwise. "Source" links go to the arXiv abstract page, the publisher PDF, or the lab's official release. "Organization" lists the primary affiliation of the first author or the lab that led the work. "Product" lists the model, system, or technique name that the paper introduced. Some early papers predate the modern convention of naming a model in the title, so the product column is blank.

For papers that have their own dedicated wiki entry, the title links to that entry.

Important papers

Scroll sideways for more →

Other papers

Scroll sideways for more →

How did the field evolve?

If you read the table top to bottom, four broad waves show up.

The first is the deep learning revival, roughly 2012 to 2016. AlexNet (Krizhevsky, Sutskever, Hinton, 2012) showed that GPUs and large labeled datasets (ImageNet) could push convolutional networks past hand-engineered features. At the ILSVRC-2012 competition AlexNet won with a top-5 test error of 15.3 percent, against 26.2 percent for the second-best entry, and the entire computer vision community switched over within months ^[1]. Word2Vec (Mikolov et al., 2013) brought the same trick to language, mapping words to dense vectors where geometric relationships matched semantic ones. The famous "king minus man plus woman equals queen" example came from this paper. GANs (Goodfellow et al., 2014) introduced adversarial training for generation, pitting a generator against a discriminator in a minimax game. DQN, A3C, and the AlphaGo line showed that deep RL could solve previously intractable games. AlphaGo beat Lee Sedol 4 to 1 in Seoul in March 2016, a series watched by more than 200 million people, in a year that most professional Go players had predicted would not come for another decade ^[2]. ResNet (He et al., 2015) made it possible to train networks hundreds of layers deep without vanishing gradients by adding identity skip connections. Most of these papers are still cited every week, and the architectures they introduced (CNNs, GANs, residual blocks, replay buffers) show up in almost every modern system.

The second is the transformer era, roughly 2017 to 2020. "Attention Is All You Need" (Vaswani et al., 2017) proposed "a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely," which both fixed the long-context problems of LSTMs and made training on TPUs and GPU clusters far more efficient ^[3]. BERT (Devlin et al., 2018) and GPT (Radford et al., 2018) split the family tree into encoders and decoders. BERT used masked language modeling to learn bidirectional context, which turned out to be ideal for classification and retrieval. GPT used autoregressive next-token prediction, which scaled much better for generation. T5 (Raffel et al., 2019) unified everything as text-to-text. GPT-3 (Brown et al., 2020) made the bet that scale alone unlocks new behaviors, and it mostly paid off: a 175B parameter model that had never been fine-tuned on a target task could often match or beat smaller fine-tuned models from a few prompted examples ^[4]. The Scaling Laws paper (Kaplan et al., 2020) gave the field a quantitative recipe for how loss should drop as parameters, data, and compute grew ^[5]. ViT (Dosovitskiy et al., 2020) brought transformers to vision and CLIP (Radford et al., 2021) bridged vision and language by training a joint embedding space on 400 million image-text pairs scraped from the web ^[6]. By the end of 2020, almost every state of the art system on every NLP benchmark used the same basic architecture.

The third is the alignment era, roughly 2021 to 2023. The bottleneck stopped being raw capability and started being making large models behave. Christiano et al. (2017) had introduced preference-based RL years earlier ^[7], but InstructGPT (Ouyang et al., 2022) showed it at GPT-3 scale: human labelers ranked model outputs, a reward model learned to imitate those rankings, and PPO fine-tuned the language model against the reward. The resulting 1.3B parameter InstructGPT was preferred to the 175B GPT-3 by human raters, "despite having 100x fewer parameters" ^[8]. ChatGPT, launched in November 2022, used the same recipe. Constitutional AI (Bai et al., 2022) replaced most of the human labels with model-generated feedback ^[9]. The model critiques its own responses against a written constitution, then revises them, and the revised pairs train the next round. This made alignment cheaper to scale and gave Anthropic a way to articulate the behavior they wanted in plain English rather than implicitly through labels. DPO (Rafailov et al., 2023) skipped the reward model entirely: it showed that a language model can be optimized directly against preference data with a simple cross-entropy objective, which is more stable and cheaper than PPO ^[10]. Most open-source RLHF pipelines have moved to DPO or one of its variants. Chain of Thought (Wei et al., 2022), ReAct (Yao et al., 2022), Reflexion, and self-rewarding methods turned chatbots into something closer to agents that can plan, call tools, and revise their own outputs ^[11]. RLHF, DPO, and Constitutional AI now ship in almost every commercial chatbot.

The fourth wave, still in progress, is the reasoning and multimodal era. GPT-4 (March 2023) added vision and dramatically improved performance on professional exams ^[12]. The Sparks of AGI paper (Bubeck et al., 2023) argued that "GPT-4's capabilities... could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system," which kicked off a long debate about how much the benchmark numbers actually measured ^[13]. Gemini (December 2023) was multimodal from day one, trained jointly on text, images, audio, and video rather than bolted together ^[14]. Llama 2 (July 2023), Mistral 7B (October 2023), and Mixtral 8x7B (January 2024) pushed open-weights performance close to the frontier, with Mixtral matching Llama 2 70B while using only 13B active parameters per token ^[15]. The Llama 3 herd (July 2024) released a 405B dense transformer with 128K context, narrowing the gap between open and closed frontier models to a few months ^[16]. DeepSeek-R1 (January 2025) showed that pure reinforcement learning, without supervised reasoning traces, can elicit emergent long-horizon thinking: the model learns to backtrack, verify, and self-correct on its own, reaching performance comparable to OpenAI o1 while being released under an MIT license ^[17]. AlphaFold 2 and 3 extended the same transformer-and-attention machinery to biology, predicting protein structures with near-experimental accuracy ^[18]. AlphaFold 3, released in 2024, also handles nucleic acids and small-molecule ligands ^[19]. John Jumper and Demis Hassabis shared half of the 2024 Nobel Prize in Chemistry for AlphaFold, with the other half awarded to David Baker for computational protein design ^[20].

Which papers are easy to underrate?

A few entries in the table do not get the same airtime as the headline architectures, but they end up doing a lot of work.

Adam (Kingma and Ba, 2014) is the optimizer that almost every model on this page was trained with, at least at some point. Variants like AdamW are now the default. The paper itself is short and unflashy.

LoRA (Hu et al., 2021) made fine-tuning large models accessible to anyone with a single GPU. By freezing the base model and learning a low-rank update, you can adapt a 7B model on a laptop. Hugging Face, Replicate, and most of the open-source fine-tuning ecosystem rely on it ^[21].

The Chinchilla paper (Hoffmann et al., 2022) corrected the original Kaplan scaling laws. Kaplan's recipe had told everyone to make models bigger; Chinchilla found that "model size and the number of training tokens should be scaled equally" for a given compute budget ^[22]. To prove it, the team trained a 70B parameter Chinchilla model on 1.4 trillion tokens using the same compute as the 280B Gopher, and Chinchilla outperformed Gopher, GPT-3, Jurassic-1, and Megatron-Turing despite being four times smaller ^[22]. Most frontier models since 2022 have followed Chinchilla-style ratios.

Whisper (Radford et al., 2022) made high-quality multilingual speech recognition free. The model was trained on 680,000 hours of weakly supervised audio scraped from the internet, covering 96 languages, and it has become the default backbone for transcription products, podcast tools, and voice agents ^[23].

The original Latent Diffusion paper (Rombach et al., 2021, published at CVPR 2022) is the actual technical foundation of Stable Diffusion ^[24]. Stability AI funded the open release a few months later. Without this paper, the consumer text-to-image boom of 2022 to 2023 probably does not happen.

How should you use this index?

If you are new to the field and want a reading order, a reasonable path is: ImageNet, AlexNet, Word2Vec, Seq2Seq, Attention Is All You Need, BERT, GPT-2, Scaling Laws, GPT-3, InstructGPT, Constitutional AI, Chinchilla, Chain of Thought, LLaMA, GPT-4 Technical Report, DPO, Llama 3, DeepSeek-R1. That gives you the spine of the modern stack in roughly chronological order, with each paper building on the previous.

If you are tracking a specific subfield, the "Type" column groups papers by area. Vision papers are easy to skim by filtering on Computer Vision. RL papers are sparser but include the most cited single results in the field (AlphaGo, MuZero, DQN, PPO). Audio is covered by WaveNet, AudioLM, MusicGen, Whisper, and SeamlessM4T. Robotics shows up under RT-2 and PaLM-E.

If you are looking for the alignment and safety literature in particular, the core spine is: Christiano et al. (2017), InstructGPT (2022), the Anthropic HH paper (2022), Constitutional AI (2022), and DPO (2023). Each of these is short, readable, and shapes how production models are trained today.

What is missing from this index?

This page is a living index. Some papers are obviously canonical and missing only because no one has written a row yet. Likely additions worth tracking: the original GAN follow-ups (DCGAN, StyleGAN), the long context line (FlashAttention, Mamba, state-space models), agent benchmarks (SWE-bench, GAIA, OSWorld), safety and evaluation work (RealToxicityPrompts, TruthfulQA, HELM), the o-series reasoning papers from OpenAI, and the Anthropic interpretability papers on dictionary learning and circuits. New rows can be added in chronological order under either table.

References

1. Krizhevsky, A., Sutskever, I., Hinton, G. (2012). ImageNet Classification with Deep Convolutional Neural Networks. NeurIPS 2012. PDF ↩
2. AlphaGo versus Lee Sedol, March 2016 (4 to 1, Seoul). Wikipedia; Silver, D. et al. (2016). Mastering the game of Go with deep neural networks and tree search. Nature 529, 484 to 489. Nature ↩
3. Vaswani, A. et al. (2017). Attention Is All You Need. arXiv:1706.03762 ↩
4. Brown, T. et al. (2020). Language Models are Few-Shot Learners. NeurIPS 2020. arXiv:2005.14165 ↩
5. Kaplan, J. et al. (2020). Scaling Laws for Neural Language Models. arXiv:2001.08361 ↩
6. Radford, A. et al. (2021). Learning Transferable Visual Models From Natural Language Supervision (CLIP). arXiv:2103.00020 ↩
7. Christiano, P. et al. (2017). Deep reinforcement learning from human preferences. arXiv:1706.03741 ↩
8. Ouyang, L. et al. (2022). Training language models to follow instructions with human feedback. arXiv:2203.02155 ↩
9. Bai, Y. et al. (2022). Constitutional AI: Harmlessness from AI Feedback. arXiv:2212.08073 ↩
10. Rafailov, R. et al. (2023). Direct Preference Optimization: Your Language Model is Secretly a Reward Model. arXiv:2305.18290 ↩
11. Wei, J. et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv:2201.11903 ↩
12. OpenAI (2023). GPT-4 Technical Report. arXiv:2303.08774 ↩
13. Bubeck, S. et al. (2023). Sparks of Artificial General Intelligence: Early experiments with GPT-4. arXiv:2303.12712 ↩
14. Gemini Team, Google (2023). Gemini: A Family of Highly Capable Multimodal Models. arXiv:2312.11805 ↩
15. Jiang, A. et al. (2024). Mixtral of Experts. arXiv:2401.04088 ↩
16. Llama Team, Meta (2024). The Llama 3 Herd of Models. arXiv:2407.21783 ↩
17. DeepSeek-AI (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948 ↩
18. Jumper, J. et al. (2021). Highly accurate protein structure prediction with AlphaFold. Nature 596, 583 to 589. Nature ↩
19. Abramson, J. et al. (2024). Accurate structure prediction of biomolecular interactions with AlphaFold 3. Nature 630, 493 to 500. Nature ↩
20. The Nobel Prize in Chemistry 2024. NobelPrize.org. Press release ↩
21. Hu, E. et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685 ↩
22. Hoffmann, J. et al. (2022). Training Compute-Optimal Large Language Models (Chinchilla). arXiv:2203.15556 ↩
23. Radford, A. et al. (2022). Robust Speech Recognition via Large-Scale Weak Supervision (Whisper). arXiv:2212.04356 ↩
24. Rombach, R. et al. (2022). High-Resolution Image Synthesis with Latent Diffusion Models. CVPR 2022. arXiv:2112.10752 ↩
25. Hochreiter, S. and Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation 9(8), 1735 to 1780. doi link
26. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P. (1998). Gradient-Based Learning Applied to Document Recognition. Proceedings of the IEEE 86(11). PDF
27. Deng, J., Dong, W., Socher, R., Li, L., Li, K., Fei-Fei, L. (2009). ImageNet: A Large-Scale Hierarchical Image Database. CVPR 2009. PDF
28. Silver, D. et al. (2017). Mastering the game of Go without human knowledge. Nature 550, 354 to 359. Nature
29. Lewis, P. et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020. arXiv:2005.11401
30. Ho, J., Jain, A., Abbeel, P. (2020). Denoising Diffusion Probabilistic Models. arXiv:2006.11239
31. Bai, Y. et al. (2022). Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. arXiv:2204.05862
32. Ramesh, A. et al. (2022). Hierarchical Text-Conditional Image Generation with CLIP Latents. arXiv:2204.06125
33. Touvron, H. et al. (2023). LLaMA: Open and Efficient Foundation Language Models. arXiv:2302.13971
34. Touvron, H. et al. (2023). Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv:2307.09288
35. Jiang, A. et al. (2023). Mistral 7B. arXiv:2310.06825