AI Wiki

Explore AI Wiki

Citation and evidence

Jim Fan

14 min full readUpdated 37 references

This article's verification

Report a problem with this article

Edit article
More

Use this article

Raw MarkdownExplore connections

Improve this page

Revision historyDiscuss this article

Topics

Embodied AINVIDIAPeopleRobotics

Cite this article

Linxi "Jim" Fan is an artificial intelligence researcher at NVIDIA, where he co-leads the Generalist Embodied Agent Research (GEAR) group with Yuke Zhu.[1] GEAR builds foundation models for agents that act in simulated and physical worlds, and is the lab behind NVIDIA's Isaac GR00T humanoid robot models.[1][6] Fan was a project lead of GR00T N1, the 2025 open foundation model for humanoid robots,[18] and before that led or co-authored a run of agent papers built around games and robot learning, among them MineDojo, which won an Outstanding Datasets and Benchmarks Paper award at NeurIPS 2022, and Voyager.[10][11][13] He received his Ph.D. from Stanford University under Fei-Fei Li and has described himself as OpenAI's first intern, from 2016.[2][5] He posts as @DrJimFan on X, where he has promoted framings such as the "Physical Turing Test".[30][31]

Education

Fan studied computer science at Columbia University's engineering school (Columbia Engineering). In April 2016 the school named him valedictorian of the class of 2016, which carried the school's Illig Medal and the valedictorian speech at Engineering Class Day on 16 May 2016. The school's profile listed his senior design project as Laminar, a C++ framework for training and deploying neural networks, and his plan to begin a computer science Ph.D. at Stanford focused on artificial intelligence.[4] As an undergraduate he worked as a research assistant in the Columbia NLP group under Michael Collins, studying kernel methods for speech recognition, and in two other Columbia labs.[2]

He did his doctorate at the Stanford Vision Lab, advised by Fei-Fei Li, from 2016 to 2021.[2][3] The Mathematics Genealogy Project records the Stanford Ph.D. as 2021, with Li as advisor.[5] His thesis was titled "Training and Deploying Visual Agents at Scale".[2]

"The Physical Turing Test: Jim Fan on Nvidia's Roadmap for Embodied AI", video from Sequoia Capital, published May 7, 2025. Watch on YouTube.

Internships and early research

Fan's website lists a series of research internships before and during his Ph.D.:[2]

PeriodOrganizationWork listed
May to September 2015Baidu Silicon Valley AI LabCo-developed Deep Speech 2, an end-to-end speech recognition system for English and Mandarin
September 2015 to March 2016Mila (Montreal)Research assistant mentored by Yoshua Bengio; analysis of the Ladder Network for semi-supervised learning
June 2016 to March 2017OpenAICo-designed World of Bits, a platform for teaching agents to use a web browser, as part of OpenAI Universe
June to September 2018Google Cloud AIBuilt SURREAL, an open-source distributed reinforcement learning framework for robot learning
June to September 2020NVIDIAProposed SECANT, a method for zero-shot generalization of visual policies

Expanded article table

The resulting papers bear this out. Fan is among the authors of the Baidu Deep Speech 2 paper,[8] the second author of "Deconstructing the Ladder Network Architecture" with Mohammad Pezeshki, Aaron Courville, Yoshua Bengio and others,[9] and third author of "World of Bits: An Open-Domain Platform for Web-Based Agents" (ICML 2017), alongside Tianlin Shi, Andrej Karpathy, Jonathan Hernandez and Percy Liang.[7] On his website he writes that he "was OpenAI's very first intern in 2016", and that World of Bits was an agent that "perceives the web browser in pixels and outputs keyboard/mouse control".[2] In a 2024 interview on Sequoia Capital's Training Data podcast he said his intern mentors at OpenAI were Andrej Karpathy and Ilya Sutskever, that World of Bits was part of the larger OpenAI Universe initiative, and that the reinforcement learning methods of the time worked on the designed tasks but did not generalize.[6]

At Stanford his first-author papers included SURREAL (CoRL 2018), RubiksNet (ECCV 2020), a video action recognition method, and SECANT (ICML 2021). He was also a co-author of the iGibson 1.0 simulation environment (IROS 2021) and MetaMorph (ICLR 2022).[2]

NVIDIA

Fan joined NVIDIA as a research scientist in December 2021.[2] His NVIDIA Research profile describes him as a Senior AI Research Scientist whose mission is "to build generalist AI agents with applications to game AI, robotics, and software automation".[3] His personal website, whose footer is dated 2023, gives his title as Senior Research Scientist and "Lead of AI Agents Initiative".[2]

GEAR's lab page describes it as "the NVIDIA GEAR group led by Dr. Jim Fan and Prof. Yuke Zhu", with a mission "to build foundation models for embodied agents in virtual and physical worlds". The page lists GR00T, Eureka, Voyager, VIMA, MineDojo and EgoScale as projects.[1] In the 2024 Sequoia interview, Fan summarized GEAR's work in three words, "we generate actions", with actions in virtual worlds meaning game AI and simulation and actions in the physical world meaning robotics. He said the team was focused on Project GR00T, which Jensen Huang had unveiled at GTC in March 2024 as NVIDIA's foundation model effort for humanoid robots.[6][17]

Descriptions of his current title vary. His X profile, as of October 2026, reads "NVIDIA Director of Robotics & Distinguished Scientist. Co-Lead of GEAR lab."[36] An August 2025 announcement for a RoboBusiness panel called him "director of AI and a distinguished research scientist at NVIDIA",[35] while the speaker listing for the RAISE Summit 2026 in Paris gave "Principal Research Scientist/Senior Research Manager".[37]

Research

Minecraft and generalist agents (2022 to 2023)

MineDojo (June 2022), with Fan as first author, is a framework built on Minecraft that combines a simulation suite of thousands of open-ended tasks with an internet-scale knowledge base of Minecraft videos, tutorials, wiki pages and forum posts. Its agent uses a pretrained video-language model as a learned reward function, so tasks can be given in free-form language without hand-designed dense rewards.[10] NeurIPS 2022 named it one of two Outstanding Datasets and Benchmarks Papers, the other being LAION-5B.[11]

VIMA (October 2022; ICML 2023), on which Fan is last author, frames many robot manipulation tasks as multimodal prompts that interleave text and images. The paper introduced a benchmark of procedurally generated tabletop tasks with more than 600,000 expert trajectories and a transformer agent that outputs motor actions autoregressively.[12]

Voyager (May 2023), led by Guanzhi Wang with Fan as a co-author, is a Minecraft agent driven by GPT-4 through black-box queries. It combines an automatic curriculum, a growing library of executable code skills, and iterative prompting that uses environment feedback, execution errors and self-verification. The authors reported that it obtained 3.3 times more unique items and unlocked key tech-tree milestones up to 15.3 times faster than the prior state of the art.[13]

LLM-written rewards and humanoid control (2023 to 2025)

Eureka (October 2023; ICLR 2024) used GPT-4 to write and evolve reward-function code for reinforcement learning. The paper reports that Eureka's rewards beat expert human-written ones on 83% of 29 environments, and it showed a simulated Shadow Hand spinning a pen.[14] DrEureka (RSS 2024) extended the idea to sim-to-real transfer, having a language model write both the reward and the domain randomization settings; its demonstrations included a quadruped walking on a yoga ball.[15] HOVER (ICRA 2025) distilled several humanoid control modes into one whole-body controller,[16] and SONIC (first posted November 2025) scaled motion-tracking training to build a general humanoid controller, scaling network size from 1.2 million to 42 million parameters and training on more than 100 million frames from 700 hours of motion capture.[21] Fan is a co-author on all four.

GR00T (2024 to present)

The GR00T N1 technical report (March 2025) names Fan and Yuke Zhu as project leads; all authors, including the leads, are listed alphabetically. GR00T N1 is a vision-language-action model with a dual-system design: a vision-language module ("System 2") interprets the scene and instructions, and a diffusion transformer ("System 1") generates motor actions in real time. It was trained on a mix of real robot trajectories, human videos and synthetic data, and deployed on the Fourier GR-1 humanoid.[18] Fan is also listed among the authors of GR00T N1.5 (June 2025) and GR00T N1.6 (December 2025).[19][20]

The two-system split matches how Fan described robot intelligence in 2024, drawing on the book Thinking, Fast and Slow: System 1 as fast, unconscious motor control and System 2 as slow, deliberate reasoning and planning. In that interview he said he hoped for a "GPT-3 moment for robotics", a research breakthrough in robot foundation models, within two to three years, and described a data strategy that mixes internet-scale data, simulation data and real robot data from teleoperation.[6][33]

World models, human video and code agents (2026)

In 2026 Fan co-authored work on world action models, human video and coding agents for robots:

DateWorkWhat it does
17 February 2026DreamZero ("World Action Models are Zero-shot Policies")A 14-billion-parameter world action model built on a pretrained video diffusion backbone, which jointly predicts future video and robot actions[22]
18 February 2026EgoScaleTrains a VLA model on 20,854 hours of action-labeled egocentric human video and reports a log-linear scaling law between human data scale and validation loss[23]
23 March 2026CaP-XA framework and benchmark for "code-as-policy" agents that control robots by writing programs; Fan is last author[24]
18 June 2026ENPIREA harness that lets coding agents run real-world robot policy improvement (automatic reset, verification, rollout and evolution); accepted to CoRL 2026, with Fan, Zhu and Guanya Shi as equal advisers[25]
30 June 2026ASPIREA coding agent that writes, debugs and refines robot control programs and saves validated fixes in a reusable skill library; Fan is one of four project leads[26]

Expanded article table

Fan called ENPIRE and ASPIRE the first two works in a series on "Physical AutoResearch".[28] When announcing ASPIRE he described it as "a new type of continual learning: 'training' is skill refinement instead of gradient descent", with the "trained model" being "a repo of sensorimotor skills instead of floating weights".[27] On 16 October 2026 he announced the open-source release of ASPIRE v1.0, writing that on the day GPT-6 Astra arrived the lab "switched the backend model and saturated all the benchmarks to ~99% over night", and that Claude Opus 5.5 "slightly edges out Astra on our robot simulation bench". He ended the post by inviting readers to "feel the Physical Bitter Lesson at home", a reference to Richard Sutton's Bitter Lesson.[29]

Public talks and framings

Physical Turing Test. Fan wrote on X: "The Physical Turing Test: your house is a complete mess after a Sunday hackathon. On Monday night, you come home to an immaculate living room and a candlelight dinner. And you couldn't tell whether a human or a machine had been there. Deceptively simple, insanely hard."[31] He built his talk at Sequoia Capital's AI Ascent 2025 around the idea; Sequoia published the recording on 7 May 2025 as "The Physical Turing Test: Jim Fan on Nvidia's Roadmap for Embodied AI".[30]

Robotics' End Game. At AI Ascent 2026, Sequoia published his talk "Robotics' End Game: Nvidia's Jim Fan" on 30 April 2026.[32] He was scheduled to give a keynote titled "Robotics: The End Game" at the MACHINA track of the RAISE Summit at Station F in Paris on 7 July 2026. The organizers' description says he argued that robotics is following the path of large language models "from pre-training to reasoning to auto-research", with world models replacing language models, egocentric video replacing teleoperation, and world action models replacing the VLA paradigm, and that he expected the Physical Turing Test to be passed within two to three years.[34]

Selected publications

YearTitleVenueFan's position
2016Deep Speech 2: End-to-End Speech Recognition in English and MandarinICMLCo-author[8]
2017World of Bits: An Open-Domain Platform for Web-Based AgentsICMLThird author[7]
2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale KnowledgeNeurIPS (Datasets and Benchmarks)First author[10]
2023VIMA: General Robot Manipulation with Multimodal PromptsICMLLast author[12]
2023Voyager: An Open-Ended Embodied Agent with Large Language ModelsTMLR (2024)Co-author[13]
2024Eureka: Human-Level Reward Design via Coding Large Language ModelsICLRCo-author[14]
2024DrEureka: Language Model Guided Sim-To-Real TransferRSSCo-author[15]
2025HOVER: Versatile Neural Whole-Body Controller for Humanoid RobotsICRACo-author[16]
2025GR00T N1: An Open Foundation Model for Generalist Humanoid RobotsarXiv technical reportProject lead[18]
2026ENPIRE: Agentic Robot Policy Self-Improvement in the Real WorldCoRLEqual adviser[25]
2026ASPIRE: Agentic /Skills Discovery for RoboticsarXiv preprintProject lead[26]

Expanded article table

Continue reading

More to read

Topics:Embodied AINVIDIAPeopleRobotics

References

  1. ^1 ^2 ^3NVIDIA Research, "GEAR: Generalist Embodied Agent Research" lab page. research.nvidia.com/...gear
  2. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9Linxi "Jim" Fan, personal website (biography, publications and experience). jimfan.me
  3. ^1 ^2NVIDIA Research, "Linxi 'Jim' Fan" researcher profile. research.nvidia.com/...linxi-jim-fan
  4. ^Melanie A. Farmer, "Top of Their Class," Columbia Engineering, 18 April 2016. engineering.columbia.edu/...top-their-class
  5. ^1 ^2Mathematics Genealogy Project, "Linxi Fan" (MGP ID 343366). mathgenealogy.org/id
  6. ^1 ^2 ^3 ^4Sequoia Capital, "Jim Fan on Nvidia's Embodied AI Lab and Jensen Huang's Prediction that All Robots will be Autonomous," Training Data podcast, 2024. sequoiacap.com/...training-data-jim-fan
  7. ^1 ^2Tianlin Shi, Andrej Karpathy, Linxi Fan, Jonathan Hernandez, Percy Liang, "World of Bits: An Open-Domain Platform for Web-Based Agents," Proceedings of the 34th International Conference on Machine Learning, PMLR 70:3135-3144, 2017. proceedings.mlr.press/...shi17a
  8. ^1 ^2Dario Amodei et al., "Deep Speech 2: End-to-End Speech Recognition in English and Mandarin," arXiv:1512.02595. arxiv.org/...1512.02595
  9. ^Mohammad Pezeshki, Linxi Fan, Philemon Brakel, Aaron Courville, Yoshua Bengio, "Deconstructing the Ladder Network Architecture," arXiv:1511.06430. arxiv.org/...1511.06430
  10. ^1 ^2 ^3Linxi Fan et al., "MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge," arXiv:2206.08853, 17 June 2022. arxiv.org/...2206.08853
  11. ^1 ^2NeurIPS, "Announcing the NeurIPS 2022 Awards," NeurIPS blog, 21 November 2022. blog.neurips.cc/...ouncing-the-neurips-2022-awards
  12. ^1 ^2Yunfan Jiang et al., "VIMA: General Robot Manipulation with Multimodal Prompts," arXiv:2210.03094 (ICML 2023). arxiv.org/...2210.03094
  13. ^1 ^2 ^3Guanzhi Wang et al., "Voyager: An Open-Ended Embodied Agent with Large Language Models," arXiv:2305.16291, 25 May 2023. arxiv.org/...2305.16291
  14. ^1 ^2Yecheng Jason Ma et al., "Eureka: Human-Level Reward Design via Coding Large Language Models," arXiv:2310.12931 (ICLR 2024). arxiv.org/...2310.12931
  15. ^1 ^2Yecheng Jason Ma et al., "DrEureka: Language Model Guided Sim-To-Real Transfer," arXiv:2406.01967 (RSS 2024). arxiv.org/...2406.01967
  16. ^1 ^2Tairan He et al., "HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots," arXiv:2410.21229 (ICRA 2025). arxiv.org/...2410.21229
  17. ^NVIDIA, "NVIDIA Announces Project GR00T Foundation Model for Humanoid Robots and Major Isaac Robotics Platform Update," NVIDIA Newsroom, 18 March 2024. nvidianews.nvidia.com/...l-isaac-robotics-platform
  18. ^1 ^2 ^3NVIDIA (Johan Bjorck et al.), "GR00T N1: An Open Foundation Model for Generalist Humanoid Robots," arXiv:2503.14734, 18 March 2025. arxiv.org/...2503.14734
  19. ^NVIDIA GEAR, "GR00T N1.5" research page, 11 June 2025. research.nvidia.com/...gr00t-n1_5
  20. ^NVIDIA GEAR, "GR00T N1.6" research page, 15 December 2025. research.nvidia.com/...gr00t-n1_6
  21. ^Zhengyi Luo et al., "SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Control," arXiv:2511.07820. arxiv.org/...2511.07820
  22. ^Seonghyeon Ye et al., "World Action Models are Zero-shot Policies," arXiv:2602.15922, 17 February 2026. arxiv.org/...2602.15922
  23. ^Ruijie Zheng et al., "EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data," arXiv:2602.16710, 18 February 2026. arxiv.org/...2602.16710
  24. ^Letian Fu et al., "CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation," arXiv:2603.22435, 23 March 2026. arxiv.org/...2603.22435
  25. ^1 ^2Wenli Xiao et al., "ENPIRE: Agentic Robot Policy Self-Improvement in the Real World," arXiv:2606.19980, 18 June 2026; project page research.nvidia.com/...enpire . arxiv.org/...2606.19980
  26. ^1 ^2Runyu Lu et al., "ASPIRE: Agentic /Skills Discovery for Robotics," arXiv:2607.00272, 30 June 2026. arxiv.org/...2607.00272
  27. ^Jim Fan (@DrJimFan), post introducing ASPIRE, X, 30 June 2026. x.com/...2072004190856212902
  28. ^Jim Fan (@DrJimFan), "ENPIRE -> ASPIRE, our 2nd work in the series for Physical AutoResearch," X, 1 July 2026. x.com/...2072337488782475390
  29. ^Jim Fan (@DrJimFan), post announcing the ASPIRE v1.0 open-source release, X, 16 October 2026. x.com/...2111128797181567466
  30. ^1 ^2Sequoia Capital, "The Physical Turing Test: Jim Fan on Nvidia's Roadmap for Embodied AI," AI Ascent 2025 talk, YouTube, 7 May 2025. youtube.com/watch
  31. ^1 ^2OfficeChai, "NVIDIA Director Of AI Jim Fan Proposes A 'Physical Turing Test'," 9 May 2025. officechai.com/...sical-turing-test-jim-fan-nvidia
  32. ^Sequoia Capital, "Robotics' End Game: Nvidia's Jim Fan," AI Ascent 2026 talk, YouTube, 30 April 2026. youtube.com/watch
  33. ^The Decoder, "Nvidia researcher Jim Fan expects 'GPT-3 moment' for robotics in the next few years," 18 September 2024. the-decoder.com/...-robotics-in-the-next-few-years
  34. ^RAISE Summit, "Keynote | Robotics: The End Game" (MACHINA 2026 session, 7 July 2026). tickets.raisesummit.com/...e-robotics-the-end-game
  35. ^Automated Warehouse, "Keynote panel to discuss commercial humanoid deployments at RoboBusiness," 5 August 2025. automatedwarehouseonline.com/...ness-keynote-panel
  36. ^Jim Fan (@DrJimFan), X profile, accessed 17 October 2026. x.com/DrJimFan
  37. ^RAISE Summit, "Jim Fan" speaker listing, RAISE Summit 2026. tickets.raisesummit.com/...jim-fan

Improve this article

Add missing citations, update stale details, or write a clearer explanation. Anyone can edit: changes go live when you publish, and every edit is checked by an editor.

3 revisions · v3 · 2,785 words · full history

Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify

Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here

Reviewer note: Independent verification 2026-10-17 (xg19 V2): biography claims and 16 papers in the publications table checked against primary sources; 4 minor fixes applied.

Cite this page: AI Wiki. "Jim Fan." aiwiki.ai, updated 16 Oct 2026, fact-checked 16 Oct 2026. CC BY 4.0. https://aiwiki.ai/wiki/jim_fan

Edit article