World action model

Scroll sideways for more →

A world action model (WAM) is a robot policy design that builds action generation on a video world model backbone rather than on a vision-language model, so that a single network jointly predicts how a scene will evolve and what actions the robot should execute. Proponents argue that because the backbone has already learned visual dynamics, how objects move when pushed, grasped, or dropped, the policy inherits a physics prior that a language-centric backbone lacks, and therefore generalizes better to unseen tasks, objects, and environments.^[1][2] The term was popularized in robotics during 2026 by NVIDIA: its GEAR research group introduced the first system explicitly named a WAM, DreamZero, in a February 2026 paper titled "World Action Models are Zero-shot Policies," and NVIDIA researcher Jim Fan promoted the concept in a widely circulated talk that spring.^[2][6][7] Whether the design displaces or merges with VLAs remained an open question as of August 7, 2026.

The label is newer than the idea. Jointly predicting video and actions, and reusing video-generation models as policy backbones, has an academic lineage that predates the WAM name and extends well beyond NVIDIA, including UniPi (2023), ByteDance's GR-2 (2024), Stanford's Unified Video Action model (2025), and Alibaba's WorldVLA (2025).^{[13][14][15][16]} Readers should therefore treat "world action model" partly as a technical category and partly as NVIDIA's framing for a research direction that several labs pursue under other names.

Definition

A WAM is a policy: given observations (typically multi-camera images plus a language instruction, sometimes proprioceptive state), it outputs executable robot actions. What distinguishes it from a vision-language-action model (VLA) is the pretrained backbone and the training objective. A VLA starts from a vision-language model that learned to describe images in text, then adds an action module trained on robot demonstrations. A WAM starts from a video world model or video diffusion model that learned to predict how scenes evolve over time, then trains video prediction and action prediction jointly, treating action as another generated modality rather than as an add-on head.^[1][2][5]

This also separates WAMs from two adjacent categories:

• A world model by itself predicts future states or video, but does not output actions; a robot system can use one for planning or synthetic data while a separate policy acts. Meta's V-JEPA 2-AC, for example, is an action-conditioned video world model that controls Franka arms by planning toward image goals through model predictive control, rather than by directly generating an action chunk from a unified policy network.^[22]
• A video prediction model generates plausible future frames but has no action interface at all. NVIDIA's August 2026 blog on WAMs makes the policy requirement explicit: the model must produce robot action trajectories, with the predicted video as a companion output of the same forward pass.^[1]

Usage is not fully settled. NVIDIA's own blogs write both "world action model" and "world-action model," and group under the label systems whose architectures differ substantially, from monolithic diffusion transformers that denoise video and action tokens together to omnimodels where an autoregressive reasoner guides a diffusion generator.^[1][5]

Origin of the term

The earliest prominent robotics use of the exact phrase identified for this article is NVIDIA's DreamZero paper, posted to arXiv on February 17, 2026 by the company's GEAR group with project leads Seonghyeon Ye, Yuke Zhu, Linxi "Jim" Fan, and Joel Jang. Its title states the thesis: "World Action Models are Zero-shot Policies."^[2] Yuke Zhu announced the release as "a World Action Model that turns video world models into zero-shot robot policies."^[26]

The concept reached a wider audience through Jim Fan's talk "Robotics' End Game" at Sequoia Capital's AI Ascent 2026 conference (the recording was published on April 30, 2026). Fan argued that VLAs devote most of their parameters to language, making them "head heavy in the wrong places," strong on semantics but weak on physics, and, as recounted in a published summary of the talk, staged a mock eulogy: "Let's all take a moment of silence for our dear friend VLAs. They've served us well. Rest in peace. Long live world action models."^[6][7] That widely shared May 2026 summary, by YK Sugi, carried the title "VLAs are dead, long live World Action Models," a phrase NVIDIA's own developer blog later quoted as shorthand for the shift.^[1][7]

NVIDIA then institutionalized the term. A June 15, 2026 developer blog by Moritz Reuss surveyed "the rise of world-action models," and an August 4, 2026 blog by Saeed Babamohamadi, "Beyond VLAs: How World Action Models Reshape Robot Manipulation," positioned the company's Cosmos 3 omnimodel as a WAM foundation.^[1][5] NVIDIA Robotics promoted the August post with the argument that VLA backbones "aren't trained to predict what happens after the gripper moves," and announced a Cosmos Labs livestream on WAMs, VLAs, and hybrid robot foundation models, dated August 13 in the blog.^[1][23][24]

Related coinages predate and surround the NVIDIA usage. Microsoft Research published Muse, a "World and Human Action Model" (WHAM), in Nature in February 2025; it jointly generates game visuals and controller actions, an analogous joint video-action formulation in the video game domain rather than robotics.^[18][19] Alibaba's WorldVLA (June 2025) called itself an "autoregressive action world model."^[14] By mid-2026 groups outside NVIDIA were adopting the WAM term directly: DriveWAM (May 2026) applies "world-action modeling" to autonomous driving, and a June 2026 paper proposing "world-language-action" (WLA) models treats the WAM as an established class to build upon.^[20][21]

Motivation: what VLAs are said to miss

The standard recipe for a language-conditioned generalist robot policy adds an action module to a pretrained vision-language model. NVIDIA's critique is that a VLM backbone learns to describe the world, not predict how it evolves: it does not learn what happens to a mug when the gripper closes or how a towel folds, so VLAs generalize well semantically but less well physically, to unseen behaviors and environments.^[1] The DreamZero paper states the same limitation as its opening premise: state-of-the-art VLAs "excel at semantic generalization but struggle to generalize to unseen physical motions in novel environments."^[2]

Independent research groups articulated versions of this motivation before the WAM label existed. The Stanford authors of the Unified Video Action model argued in 2025 that videos provide rich scene information for action prediction and actions provide dynamics information for video prediction, while noting that earlier video-generation-based policies had struggled to match direct policy learning in action accuracy and inference speed.^[13] WorldVLA's authors found that adding a world-model objective improved action generation because the model learns "the underlying physics of the environment."^[14] GR-2 pretrained on 38 million internet video clips explicitly "to capture the dynamics of the world" before fine-tuning for video generation and action prediction.^[15]

NVIDIA's papers and blogs list three practical benefits claimed for the WAM approach, all of which are vendor-reported claims rather than independently established findings:^[1][2]

1. Learning from diverse data. A VLA trained by imitation learning often needs many near-identical demonstrations of one task; a model learning dynamics can extract signal from any interaction data, which its proponents argue lowers data-collection cost.
2. Open-world generalization. Physics is claimed to be more general than semantics: the way an object falls does not change with the object, so learned dynamics carry into scenes and motions absent from training.
3. Cheaper adaptation to new robots. A backbone that already models physical interaction is claimed to need less task-specific data to specialize to a new arm or gripper, including from video-only demonstrations without action labels.

How world action models work

Published WAM-style systems share a pattern: take a large pretrained video generation backbone, then post-train it so that robot actions become an output modality generated jointly with future video. The implementations differ in mechanism.

DreamZero starts from Wan2.1-I2V-14B-480P, an open 14-billion-parameter image-to-video diffusion model, and turns it into an autoregressive video diffusion policy that denoises video and action tokens together inside one diffusion transformer. There is no separate inverse-dynamics module; the action chunk and the predicted future frames come from the same denoising process, conditioned on visual context, a language instruction, and proprioceptive state.^[2][3][4] Because generating video is expensive, the team built DreamZero-Flash, a set of model and system optimizations the paper reports as a 38x inference speedup, enabling closed-loop control at roughly 7 Hz from a 14B model.^[2]

Cosmos 3, NVIDIA's open "omnimodel" world foundation model launched June 1, 2026, takes a different route to the same end. Its Mixture-of-Transformers architecture pairs an autoregressive reasoning transformer, which produces discrete tokens such as text, with a diffusion transformer that generates continuous modalities: image, video, audio, and action, all synthesized through iterative denoising. Action is a native output modality alongside video rather than a bolted-on head.^[1][8] NVIDIA reports the training corpus for the 16B Nano model at roughly 767 million images, 348 million videos, and 8 million action samples spanning robot manipulation, autonomous driving, camera motion, and egocentric motion.^[1]

Cosmos Policy (January 2026), from Stanford and NVIDIA researchers including Chelsea Finn and Shuran Song, showed a deliberately simple variant: adapt the earlier Cosmos-Predict2 video model into a robot policy through a single stage of post-training on target-platform demonstrations, with no architectural modifications, representing actions as latent frames inside the same video denoising interface.^[17]

A characteristic property follows from the joint formulation: the policy "imagines while it acts." When a WAM outputs an action chunk, it can simultaneously output the video its cameras would see if those actions were executed, because both come from the same generative model at the same time.^[1][2]

DreamZero results

DreamZero is the evidential centerpiece of the WAM argument, and its numbers are NVIDIA's own. The paper trains on roughly 500 hours of teleoperation data collected on the AgiBot G1 robot plus the open DROID dataset, a Stanford-led multi-institution robot manipulation corpus collected on Franka Panda arms, and reports:^[2][11][12]

• On AgiBot G1 evaluations, 62.2 percent average task progress versus 27.4 percent for the best pretrained VLA baseline, despite those baselines (Isaac GR00T N1.6 and Physical Intelligence's pi-0.5) being pretrained on thousands of hours of cross-embodiment robot data.
• On tasks entirely absent from training, such as untying shoelaces and shaking hands, 39.5 percent average task progress versus 16.3 percent for pretrained VLA baselines; from-scratch VLAs scored below 1 percent.
• On a DROID-Franka setup (80 rollouts across 40 tasks), 49 percent task progress and 22.5 percent success, versus 31 percent and 12.5 percent for an internally trained GR00T N1.6-DROID checkpoint and 33 percent and 7.5 percent for the publicly released pi-0.5-DROID.
• Cross-embodiment transfer from video-only data: 12 minutes of human video or 20 minutes of another robot's video yielded a relative improvement of over 42 percent on unseen-task performance, and 30 minutes of play data sufficed for few-shot adaptation to a new embodiment while retaining zero-shot generalization.

One partially external data point exists. NVIDIA's June 2026 blog reproduces an April 2026 snapshot of RoboArena, a distributed real-world benchmark for generalist policies run across participating institutions, showing DreamZero at 1750 against 1622 for pi-0.5, 1592 for pi-FAST, and 1475 for pi-0. The blog itself cautions that this is "not proof that WAMs are the better default, but... a positive signal for their potential."^[5] As of August 7, 2026, no independent replication of the paper's head-to-head comparisons had been identified.

Cosmos 3 policy models

NVIDIA's productized WAMs are the Cosmos 3 Policy DROID checkpoints, post-trained from the Cosmos 3 omnimodel for the DROID platform, a Franka Panda arm with a Robotiq gripper.^[1][11] Given a language instruction and multi-camera observations, they generate robot action trajectories, and because post-training removes nothing from the omnimodel, the policy checkpoints can still reason in text and generate video.^[1][9][10]

Scroll sideways for more →

Both repositories tag themselves "world action model" on Hugging Face.^[9][10] For the Edge policy, NVIDIA reports operation at 640x360 observation resolution, generating 32 actions per inference on Jetson Thor with real-time control at 15 Hz; the model card's own timing tables show that budget being met only on some Jetson configurations, so the headline rate should be read as a best-case vendor figure.^[1][10]

As evidence that the world-model prior itself helps, NVIDIA's blog cites a controlled comparison from the Cosmos 3 technical report: two DROID policies trained with the same recipe, data, and compute, one initialized from a base checkpoint and one from an omni checkpoint trained on multi-domain action data. The omni initialization raised success on the RoboLab simulation benchmark from 28.1 percent to 36.8 percent.^[1] Like the DreamZero results, this is a vendor-run evaluation.

Models associated with the paradigm

The table lists verifiable systems that either call themselves world action models or are commonly grouped into the joint video-action prediction line the term describes. Inclusion reflects the published record, not a claim that the authors endorse NVIDIA's label.

Scroll sideways for more →

NVIDIA's June 2026 survey blog additionally lists industry systems such as Ant Group's LingBot-VA and Mimic Robotics' mimic-video as video-backbone policies; these are included there on the vendor's account and are not independently detailed here.^[5]

Relationship to VLAs and hybrid models

Despite the "VLAs are dead" rhetoric, the practical picture in 2026 is coexistence, including inside NVIDIA. The company's own GR00T N1.7 (April 2026) is a vision-language-action model, and its Alpamayo 2 Super driving model (August 2026) is a reasoning VLA built on the Cosmos 3 Super Reasoner.^[27][28] NVIDIA's August 2026 livestream announcement frames the question as WAMs, VLAs, "and hybrid robot foundation models" rather than a settled succession.^[24]

Hybridization is also where much of the non-NVIDIA literature is heading. WorldVLA and its successor line fold world-model objectives into a VLA.^[14] The June 2026 world-language-action (WLA) proposal explicitly tries to conjoin "the world modeling interface... as in the world-action model" with the language reasoning of VLAs, using an autoregressive backbone instead of the bidirectional diffusion transformer typical of WAMs, precisely because long-horizon tasks benefit from textual subtask reasoning that pure video-action models lack.^[21] In embodied AI terms, the open question is not whether world modeling helps a policy but where it should live: in the pretraining backbone (the WAM position), in an auxiliary objective (the WorldVLA position), or in a separate planning model (the V-JEPA 2-AC position).^[14][21][22]

Limitations and critiques

Compute and latency. A WAM carries a full generative world model, not just an action head, and is larger than compact VLAs; NVIDIA's blog concedes the size penalty, and DreamZero needed a reported 38x inference optimization to reach 7 Hz, a rate hierarchical VLA systems exceed by an order of magnitude in their low-level controllers.^[1][2] Video generation also spends capacity on pixels that may be irrelevant to control.

Language and long-horizon reasoning. The WLA authors argue that diffusion-based WAMs inherit weak language reasoning relative to VLAs, limiting complex long-horizon tasks; the UVA authors earlier documented that video-generation-based policies had historically lagged direct policy learning on action accuracy and inference speed, which their decoupled decoding was designed to fix.^[13][21]

Vendor-dominated evidence. Nearly every head-to-head number supporting the WAM claim, DreamZero's evaluations, the RoboLab ablation, the Edge policy's control rates, comes from NVIDIA papers, model cards, or blogs, evaluated on setups NVIDIA chose. The RoboArena snapshot is the closest thing to external evidence and is a single dated leaderboard reading reproduced in a vendor blog.^[2][5] The general reproducibility problems documented for VLA evaluation, inconsistent tasks, success criteria, and hardware, apply with equal force here.

Terminological instability. "World action model" competes with "world-action model," "action world model," "video-action policy," and the unrelated-in-domain WHAM, and the boundary with VLAs blurs as hybrids multiply. A model's label is a weaker guide to its design than what it actually predicts.

Access. Openness varies within the paradigm: the Cosmos 3 policy checkpoints ship under the permissive OpenMDW-1.1 license, but the DreamZero-DROID checkpoint that anchors the research claims is released under CC BY-NC 4.0, which excludes commercial use.^[4][9][25]

See also

• Vision-language-action model
• World model
• NVIDIA Cosmos 3
• Robot foundation model
• Isaac GR00T
• Physical AI
• Robot learning
• V-JEPA

References

1. ^Babamohamadi, Saeed. "Beyond VLAs: How World Action Models Reshape Robot Manipulation." NVIDIA Technical Blog, August 4, 2026. developer.nvidia.com/...reshape-robot-manipulation
2. ^Ye, Seonghyeon, et al. "World Action Models are Zero-shot Policies." arXiv:2602.15922, February 17, 2026. arxiv.org/...2602.15922
3. ^"DreamZero: World Action Models are Zero-shot Policies." Project page, 2026. dreamzero0.github.io
4. ^"GEAR-Dreams/DreamZero-DROID." Hugging Face model card, 2026. huggingface.co/...DreamZero-DROID
5. ^Reuss, Moritz. "Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models." NVIDIA Technical Blog, June 15, 2026. developer.nvidia.com/...ise-of-world-action-models
6. ^Sequoia Capital. "Robotics' End Game: Nvidia's Jim Fan." AI Ascent 2026 talk, YouTube, published April 30, 2026. youtube.com/watch
7. ^Sugi, YK. "VLAs are dead, long live World Action Models: a summary of Jim Fan's Robotics End Game talk." May 8, 2026. eventual.ai/...-dead-long-live-world-action-models
8. ^NVIDIA Newsroom. "NVIDIA Launches Cosmos 3, the Open Frontier Foundation Model for Physical AI." June 1, 2026. nvidianews.nvidia.com/...ion-model-for-physical-ai
9. ^"nvidia/Cosmos3-Nano-Policy-DROID." Hugging Face model card, 2026. huggingface.co/...Cosmos3-Nano-Policy-DROID
10. ^"nvidia/Cosmos3-Edge-Policy-DROID." Hugging Face model card, 2026. huggingface.co/...Cosmos3-Edge-Policy-DROID
11. ^Khazatsky, Alexander, et al. "DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset." Robotics: Science and Systems XX, 2024. roboticsproceedings.org/...p120
12. ^DROID project. "DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset." droid-dataset.github.io
13. ^Li, Shuang; Gao, Yihuai; Sadigh, Dorsa; Song, Shuran. "Unified Video Action Model." arXiv:2503.00200, 2025. arxiv.org/...2503.00200
14. ^Cen, Jun, et al. "WorldVLA: Towards Autoregressive Action World Model." arXiv:2506.21539, June 2025. arxiv.org/...2506.21539
15. ^Cheang, Chi-Lam, et al. "GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation." ByteDance Research, arXiv:2410.06158, October 2024. arxiv.org/...2410.06158
16. ^Du, Yilun, et al. "Learning Universal Policies via Text-Guided Video Generation." arXiv:2302.00111, 2023. arxiv.org/...2302.00111
17. ^Kim, Moo Jin, et al. "Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning." arXiv:2601.16163, January 22, 2026. arxiv.org/...2601.16163
18. ^Microsoft Research. "Advancing game ideation with Muse: the first World and Human Action Model (WHAM)." February 2025. microsoft.com/...el-designed-for-gameplay-ideation
19. ^"World and Human Action Models towards gameplay ideation." Nature, February 2025. nature.com/...s41586-025-08600-3
20. ^Shi, Chen, et al. "DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving." arXiv:2605.28544, May 2026. arxiv.org/...2605.28544
21. ^Yang, Yi, et al. "World-Language-Action Model for Unified World Modeling, Language Reasoning, and Action Synthesis." arXiv:2606.05979, June 2026. arxiv.org/...2606.05979
22. ^Meta AI. "V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning." arXiv:2506.09985, June 2025. arxiv.org/...2506.09985
23. ^NVIDIA Robotics (@NVIDIARobotics). "VLAs can understand a robot's instructions, but their backbones aren't trained to predict what happens after the gripper moves..." X post, August 5, 2026. x.com/...2085071484737773661
24. ^NVIDIA Robotics (@NVIDIARobotics). "What comes after vision-language-action models in robotics?..." X post, August 6, 2026. x.com/...2085407590494634375
25. ^OpenMDW. "OpenMDW License Version 1.1." 2026. openmdw.ai/...1-1
26. ^Zhu, Yuke (@yukez). "New paper from GEAR team @NVIDIARobotics We released DreamZero, a World Action Model that turns video world models into zero-shot robot policies..." X post, February 2026. x.com/...2019096072690553112
27. ^NVIDIA. "NVIDIA Isaac GR00T N1.7: Open Reasoning VLA Model for Humanoid Robots." Hugging Face blog, April 2026. huggingface.co/...gr00t-n1-7
28. ^NVIDIA Blog. "NVIDIA Alpamayo 2 Super, the Frontier Open Model for Robotaxis and Autonomous Vehicles, Now Available for Commercial Use." August 4, 2026. blogs.nvidia.com/...super-open-model-now-available