Jakob Uszkoreit
Last reviewed
Jun 5, 2026
Sources
21 citations
Review status
Source-backed
Revision
v2 ยท 2,174 words
Improve this article
Add missing citations, update stale details, or suggest a clearer explanation.
Suggest edit
CtrlK
Last reviewed
Jun 5, 2026
Sources
21 citations
Review status
Source-backed
Revision
v2 ยท 2,174 words
Add missing citations, update stale details, or suggest a clearer explanation.
Jakob Uszkoreit is a German computer scientist and machine learning researcher, best known as one of the eight co-authors of the 2017 paper "Attention Is All You Need," which introduced the Transformer architecture [1][2]. He spent more than a decade at Google, where he worked on Google Translate and conducted deep learning research at Google Brain, before leaving in 2021 to co-found Inceptive, a biotechnology company that applies deep learning to the design of RNA-based medicines [3][4]. He is Inceptive's chief executive officer [4][5].
He is the son of the computational linguist Hans Uszkoreit; the two are sometimes confused, but Jakob Uszkoreit is a distinct researcher whose work centers on deep learning rather than his father's symbolic and rule-based traditions in computational linguistics [2]. Accounts of the Transformer's development note that his father, himself a computational linguist, was initially skeptical of the idea that attention without recurrence could be sufficient for language [1].
Uszkoreit's research career has focused on natural language processing, attention mechanisms, and the application of large-scale deep learning to language and vision [2][6]. He is a co-author of two papers that are widely cited as foundational to the modern Transformer era: "Attention Is All You Need," which introduced the architecture for sequence transduction, and "An Image Is Worth 16x16 Words," which adapted it to computer vision as the Vision Transformer [5][7]. According to scholarly indices, his publications have accumulated more than ten thousand citations [6].
Since 2021 his work has shifted from language to biology. At Inceptive he has promoted the idea of "biological software," the notion that medicines might be specified by their intended behavior and then compiled into molecules, by analogy with how source code is compiled into programs [3][8]. In January 2025 he accepted the Global Swiss AI Award on behalf of the eight authors of "Attention Is All You Need," recognition of the team's role in launching the modern era of generative AI [14][15].
Uszkoreit studied at the Technische Universitat Berlin (Technical University of Berlin), where he earned a Master's degree in computer science and mathematics in 2007 [2]. He interned at Google Research in 2006 and 2007 before joining the company full time [2]. Before his full-time move to Google he also worked briefly as a software engineer at Acrolinx, a Berlin-based company that develops software for checking and improving technical writing [16].
Uszkoreit joined Google in 2008 and worked there until 2021 [2][3]. He began at the company's Berlin office in March 2008, and his tenure spanned roughly thirteen years [16]. In his early years he contributed to Google Translate during the period when the service relied on statistical machine translation, working on language modeling and translation systems [2][5][16].
He later moved into deep learning research at Google Brain and Google Research, where he led teams working on natural language understanding [2][5]. Official conference biographies credit him with building the language understanding components behind the Google Assistant and a number of Google Search features [5][16]. By the end of his time at the company he held the position of senior staff software engineer [14][16]. Over this period his work increasingly centered on attention as an alternative to recurrent and convolutional approaches for modeling sequences [1][9].
An early step toward that direction was the 2016 paper "A Decomposable Attention Model for Natural Language Inference," co-authored with Ankur Parikh, Oscar Tackstrom, and Dipanjan Das and presented at EMNLP 2016 [9]. The model used attention to break natural language inference into independently solvable subproblems, making it highly parallelizable and reaching competitive results on the SNLI benchmark with far fewer parameters than prior systems and without relying on word order [9]. The emphasis on parallelizable, attention-based computation foreshadowed the Transformer [1][2].
In 2017 Uszkoreit was one of eight authors of "Attention Is All You Need," along with Ashish Vaswani, Noam Shazeer, Niki Parmar, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin [1]. The paper was published at the 31st Conference on Neural Information Processing Systems (NeurIPS 2017), with a preprint released on June 12, 2017, and all eight authors are listed as equal contributors with the author order randomized [1]. It proposed dispensing with recurrence and convolution entirely and building a sequence transduction model based solely on self-attention and feedforward layers [1].
Accounts of the project credit Uszkoreit with an early hypothesis that attention alone, without recurrence, could suffice for machine translation, an idea that gave the paper its title [1][2]. He is described as having proposed replacing recurrent networks with self-attention and as having started the internal effort to evaluate that idea, a hypothesis that ran against conventional wisdom at the time [1]. He is also reported to have chosen the name "Transformer" because he liked the sound of the word; an internal design document circulated under the title "Transformers: Iterative Self-Attention and Processing for Various Tasks," it included illustrations referencing the Transformers franchise, and the team referred to itself as "Team Transformer" [1]. The decomposable attention work he had co-authored the previous year is cited as part of the lineage that led to the architecture [1][9].
The Transformer became the basis for most subsequent large language models and for much of the generative AI systems built afterward [1][10]. As with all eight authors, Uszkoreit later left Google to pursue work at other companies or to found a startup [1].
Beyond the original Transformer paper, Uszkoreit contributed to several follow-up efforts that extended or generalized the architecture. He was a co-author of "Image Transformer," presented at the International Conference on Machine Learning (ICML 2018), which recast image generation as an autoregressive sequence problem and restricted self-attention to local neighborhoods so that larger images could be modeled [17]. He also co-authored "Universal Transformers," presented at the International Conference on Learning Representations (ICLR 2019), which combined the Transformer with a recurrent, adaptive-computation mechanism to address tasks where the standard model failed to generalize [18]. He appears as a co-author on "Tensor2Tensor for Neural Machine Translation" (2018), describing the open-source library that included a reference implementation of the Transformer [19].
After the original Transformer work, Uszkoreit contributed to extending the architecture beyond language. He was a co-author of "An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale," presented at ICLR 2021, which introduced the Vision Transformer by treating an image as a sequence of fixed-size patches and applying a standard Transformer encoder [7]. He also appears as a co-author on related Google vision papers from the same period, including the all-MLP "MLP-Mixer" architecture [6].
In July 2021 Uszkoreit left Google and co-founded Inceptive, also referred to by the legal name Inceptive Nucleics, with Rhiju Das, a Stanford biochemistry professor whose research focuses on RNA [3][5][8]. The two met at a conference in Palo Alto and decided to form a company [3]. Uszkoreit has said that several events in late 2020 prompted the move, including the birth of his daughter during the COVID-19 pandemic, the strong CASP14 results of AlphaFold 2 (which used deep learning methods related to his own), and the reported efficacy of mRNA COVID-19 vaccines [4][8]. He described turning to biological applications of deep learning as feeling like "almost a moral obligation" [4].
Inceptive develops "sequence-based medicines," including messenger RNA (mRNA), small interfering RNA (siRNA), antisense oligonucleotides (ASOs), and peptides, where the molecule's sequence largely determines its therapeutic function [11]. The company trains deep learning models on large, heterogeneous datasets spanning sequence, function, and structure, and pairs them with in-house wet-lab experiments in iterative design cycles [11]. Uszkoreit frames the goal as "enabling a new generation of medicines, reminiscent of software, but running on our cells," with molecules specified by desired behavior and then compiled into chemical descriptions [8][11]. He has repeatedly identified the scarcity of suitable biological data as the central obstacle [8].
Inceptive maintains its main wet-lab operations in Palo Alto, California, with additional presence in Berlin and Zurich, and the company has reported a distributed workforce of roughly fifty people, with people also based in cities such as London and Vancouver [3][11]. Its earliest backing came from a roughly $20 million seed round led by Vijay Pande at Andreessen Horowitz; Pande, who had served on Das's doctoral thesis committee at Stanford, agreed to fund the round shortly after meeting the founders [3]. In total the company has reported raising about $120 million across two early rounds [3].
The company has partnered with pharmaceutical firms, including at least one major European drugmaker for an infectious-disease mRNA vaccine program [12][13]. In September 2023 it announced a $100 million Series A round co-led by NVIDIA's NVentures and Andreessen Horowitz, with participation from Obvious Ventures and Section 32, reported to value the company at more than $300 million [12][13].
In June 2026 Inceptive entered a strategic collaboration with Alnylam Pharmaceuticals, a company that pioneered RNA interference (RNAi) therapeutics, to apply Inceptive's foundation models to the design of siRNA and other nucleic-acid medicines [20][21]. Announced on June 3, 2026, the agreement provided Inceptive with $30 million in upfront consideration, consisting of cash and a purchase of Inceptive equity, and made the company eligible for preclinical, regulatory, and commercial sales milestones that could bring the total value to as much as $2 billion [20][21]. The collaboration pairs Alnylam's RNAi platform and more than two decades of proprietary siRNA data with Inceptive's generative AI, and its scope includes target modeling, in-silico exploration of novel chemical modifications, and prediction of preclinical performance [20][21]. Uszkoreit said that most drug design still works through trial and error, "testing thousands of molecules and hoping something sticks," and described Alnylam's platform as "an ideal match for AI" [20].
| Year | Paper | Venue | Note |
|---|---|---|---|
| 2016 | A Decomposable Attention Model for Natural Language Inference | EMNLP 2016 | Parallelizable attention for NLI; precursor to the Transformer [9] |
| 2017 | Attention Is All You Need | NeurIPS 2017 | Introduced the Transformer architecture [1] |
| 2018 | Image Transformer | ICML 2018 | Autoregressive image generation with local self-attention [17] |
| 2018 | Tensor2Tensor for Neural Machine Translation | Workshop / arXiv | Open-source library with a reference Transformer implementation [19] |
| 2019 | Universal Transformers | ICLR 2019 | Recurrent, adaptive-computation generalization of the Transformer [18] |
| 2021 | An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale | ICLR 2021 | Introduced the Vision Transformer [7] |
| 2021 | MLP-Mixer: An all-MLP Architecture for Vision | NeurIPS 2021 | All-MLP alternative to attention for vision [6] |
On February 7, 2025, the Mindfire Foundation presented the Global Swiss AI Award to the eight authors of "Attention Is All You Need," at the Davos Town Hall during the week of the World Economic Forum [14][15]. It was the first time the award had been dedicated to a team rather than an individual, on the reasoning that major achievements in AI are increasingly the work of groups [14]. Uszkoreit attended in Davos and accepted the prize on behalf of the team [14][15]. The award cited the Transformer architecture as the foundation for systems such as GPT and BERT and for applications across natural language processing, computer vision, and scientific modeling [14][15].
As of 2026 Uszkoreit is the co-founder and chief executive officer of Inceptive, where he leads the company's effort to combine deep learning with biochemistry for the design of RNA medicines [4][5][11]. He continues to speak publicly about machine learning and its application to biology, including at industry conferences and in interviews with technology and pharmaceutical press [4][5][8].