AW-0

AW-0 is a robotic learning framework introduced in the October 2026 research paper Agentic World Action Models. It combines the generation of short skill programs, predicted future video, and robot actions in one model. The work comes from researchers affiliated with the University of California, Berkeley, NVIDIA, and the University of Pennsylvania.^[1][2]

AW-0 belongs to the world action model family: it learns to predict how a scene changes while producing actions for a robot. Its added code stream represents a task as reusable skill calls. In the default execution mode, that code guides video and action generation; a separate hybrid mode can execute selected generated programs through perception and control APIs.^[1][2]

Architecture

The framework has three transformer experts connected by joint attention. Each expert retains its own attention projections, layer normalization, and feed-forward layers. Information flows in the order code, video, then action: the video expert can attend to the code, and the action expert can attend to both. The input includes camera observations, a language instruction, the robot state, and a compressed memory of earlier program slices.^[1]

The streams undergo denoising together. Code tokens use masked-token prediction, while video and actions use flow matching. A generated program therefore conditions the predicted scene and motor commands during generation, rather than being added as an explanation after an action has been chosen.^[1]

The released implementation uses a Cosmos-Predict2.5-2B video expert, a Wan2.2 variational autoencoder, an SDAR-1.7B code-expert body, and a separate diffusion action head. A frozen Qwen3-VL vision tower supplies visual information to the code expert. The paper describes the code generation procedure as LLaDA-style masked-token diffusion. These component identities do not imply a single published total parameter count for the complete system.^[1][3][4]

Skill programs and annotation

Training requires demonstrations paired with programs for the subtask taking place in each time window. The annotation pipeline first samples demonstrations and writes programs against predefined primitive APIs. A coordinating agent identifies recurring routines and turns them into reusable skills. After the library is frozen, the pipeline reuses cached programs when the observed contact and motion pattern matches; otherwise it requests a new annotation.^[1][2]

The resulting library contains 50 skills: 23 for YAM robot demonstrations and 27 for egocentric human demonstrations. A skill is a Python function over lower-level robot primitives. Examples include folding a cloth edge, transporting an object with both hands, pouring into a target, and wiping a surface. A function can take different object names and motion arguments, allowing the same routine to label multiple demonstrations.^[1][2]

During training, the code stream contains a conditioning prompt and an active program slice for the current subtask. The model supervises the slice and uses a compact memory of prior slices to represent progress through the episode. This supplies task structure beyond what a single camera frame reveals.^[1]

Default and hybrid execution

The default policy executes the action expert's predicted chunk. The generated code acts as a plan that the other experts read; the robot does not directly run that program in this mode.^[1][3]

Hybrid execution considers the generated slice at each replanning step. It can execute code through a library of perception and control primitives when the slice passes checks for task stage, dependencies, syntax, permitted calls, and nonempty work. Otherwise, it falls back to the action chunk. Passing these checks does not establish that a program has correct arguments or will complete the task.^[1][2]

The simulation implementation uses perception components including Molmo, SAM 3, and Contact-GraspNet alongside RoboTwin's control interfaces. The paper states that evaluation programs and primitives do not access privileged simulator state. Its demonstration collection does use privileged object states and annotated grasps to make collection more efficient.^[1]

Training data

The authors pretrain AW-0 on bimanual robot demonstrations and egocentric human manipulation data. Both sources provide instructions and aligned program slices. The paper reports the following corpus statistics.^[1]

YAM recordings use three RGB cameras. Human demonstrations use one head-mounted camera, with its view replicated across the model's three input slots; tracked hand motion is retargeted to dexterous-hand joint targets. The framework uses a shared 64-dimensional state and action representation for the real-world data, while its RoboTwin implementation uses the simulator's 14-dimensional joint representation.^[1][3]

For downstream tasks, the pipeline annotates task demonstrations under the frozen skill library, followed by joint fine-tuning on program, video, and action targets. This is a demonstration-based training procedure related to imitation learning; the reported system does not learn continuously from its own deployed interactions.^[1]

Reported evaluations

The results below are the authors' experiments. The paper compares particular trained configurations and evaluation protocols; the numbers are not a general ranking of vision-language-action models or a guarantee of performance on other robots.^[1]

RoboTwin 2.0 simulation

The paper evaluates 50 tasks with 50 rollouts per task. Training uses 50 clean demonstrations per task. The clean setting has controlled scene conditions; the random setting varies backgrounds, textures, lighting, tabletop height, and distractors.^[1]

The default configuration has the highest average in this comparison, but its performance drops substantially under scene randomization. Its 20.6% random-setting success rate is below pi0.5's 27.7%. The hybrid configuration performs better in that setting while scoring lower in clean scenes.^[1]

A qualification in the paper's Appendix J.2 concerns training data: the default configuration uses demonstrations collected with RoboTwin's native motion planner, while the hybrid configuration uses code-as-policy demonstration collection. The comparison therefore changes both data collection and execution. It does not isolate the effect of switching execution modes on an otherwise identical simulation policy.^[1]

The released evaluation guide has a separate default of 100 episodes per task and setting for a complete benchmark run. That software default should not be confused with the paper's 50-rollout protocol.^[1][5]

Physical robot experiments

The authors evaluate eight manipulation tasks across a YAM station with parallel grippers and a Tianji-Sharpa platform with dexterous hands. The tasks include arranging fruit, packing a backpack, replacing a lightbulb, folding towels, dealing cards, and manipulating a scoreboard. They use 100 demonstrations per task for post-training and evaluate three training seeds with 16 rollouts each.^[1]

The metric is an average task-progress rubric score expressed as a percentage. It is not the fraction of complete successes used in the simulation table.^[1]

On the YAM towel-folding task, the authors also report a hybrid score of 90.6%, compared with 84.4% for action-chunk execution using the same post-trained model. Real-world hybrid evaluation was limited to YAM; the paper does not establish the same result for dexterous-hand code execution.^[1][2]

Released resources and reproduction

The official repository includes pretraining and post-training code, code annotation tools, RoboTwin evaluation, and real-robot support. It links a pretrained checkpoint and a RoboTwin post-training release with separate default and hybrid weights. The pretrained checkpoint is a starting point for task-specific post-training, rather than a ready-to-use policy for an arbitrary robot or instruction.^[3][4][5]

The release does not include the pretraining corpora or the Cosmos-Predict2.5-2B initialization checkpoint used to start pretraining. Reproducing the complete pretraining run from scratch therefore requires resources beyond the published code and AW-0 checkpoint.^[3][6]

Installation documentation specifies Python 3.10, CUDA 12.8, PyTorch 2.7.1, and H100-class GPUs; training scripts default to eight GPUs per node. It also lists separately downloaded components, including the video autoencoder, instruction encoder, code-model body, and vision tower. These are the maintainers' documented environment and defaults, not a verified minimum hardware requirement.^[6]

The real-world guide includes a YAM model server and robot-side client. It documents training for Tianji-Sharpa but states that a corresponding deployment implementation is not included. Tactile data in the Tianji-Sharpa recordings supports annotation tools, rather than serving as an input to the learned model.^[7]

The repository uses the BSD 2-Clause license. The pretrained model card also declares BSD 2-Clause and states that external components retain their own licenses.^[4][8]

Limitations

AW-0 depends on the quality of its automatic annotations and the coverage of its fixed skill library. A program may choose a suitable skill but bind it to the wrong object or use incorrect arguments. The authors' examples also include unstable grasping, misaligned insertion, tool collisions, and loss of contact.^[1][2]

Visual distribution shift remains a substantial weakness in the reported simulation results. Hybrid execution can use more structured control, but errors in its perception and control primitives also introduce failures. The authors identify arbitration based on expected execution reliability as future work; the current checks largely establish whether code is admissible, rather than whether it is preferable to the predicted action.^[1]

The research demonstrates a particular combination of program generation, world modeling, and learned control. Its published experiments do not establish unrestricted autonomous manipulation, continual self-improvement, or reliable operation across arbitrary robotics platforms.^[1]

Continue reading

• NvidiaNvidia Corporation is an American technology company that designs graphics processing units (GPUs) and is the world's dominant supplier of hardware for artificial intelligenceLinked in the introduction
• WWorld action modelA world action model (WAM) is a robot policy design that builds action generation on a video world model backbone rather than on a vision-language model, so that a single network jointly predicts how a scene…Mentioned twice above
• RRoboticsRobotics is the interdisciplinary study and engineering of robots: embodied machines whose mechanisms, sensors, computation, and actuators are organized to perform tasks in the physical world.Also in Robotics
• VVision-language-action modelA vision-language-action model (VLA model, or VLA) is a machine-learning model for robotics that uses visual observations and a natural-language instruction to generate actions a robot can execute.Also in Robotics
• IImitation LearningImitation learning is a family of methods for learning sequential behavior from demonstrations.Also in Robotics
• AAR-WAMAR-WAM is a research system for robotic manipulation that connects a high-level agent to a compact world action model.Also in World Models

More to read

• NVIDIA Cosmos
• V-JEPA 2
• Odyssey-3
• Lumo-2
• Inverse graphics
• Physis-Lang

Topics:AI ModelsAI ResearchRoboticsWorld Models

References

1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21 ^22 ^23 ^24 ^25Liu, Zhuoyang, et al. "Agentic World Action Models". arXiv:2610.30136v1, October 21, 2026. Sections 2-3 and Appendices B, J, and L.
2. ^1 ^2 ^3 ^4 ^5 ^6 ^7AW-0 authors. "Agentic World Action Models," official project page. Architecture, demonstrations, evaluation tables, and limitations. Accessed November 4, 2026.
3. ^1 ^2 ^3 ^4 ^5ZhuoyangLiu2005. "AW-0: Agentic World Action Models," official repository README. Release scope, implementation, and model links. Accessed November 4, 2026.
4. ^1 ^2 ^3miniFranka. "AW-0 pretrained checkpoint (yam + ego)". Model card and component licenses. Accessed November 4, 2026.
5. ^1 ^2AW-0 maintainers. "RoboTwin," evaluation and training guide. Accessed November 4, 2026.
6. ^1 ^2AW-0 maintainers. "Installation". Environment requirements and checkpoint availability. Accessed November 4, 2026.
7. ^AW-0 maintainers. "Real-world robots: YAM and Tianji + Sharpa". Supported training and deployment paths. Accessed November 4, 2026.
8. ^Liu, Zhuoyang. "BSD 2-Clause License," agentic-wam repository. Copyright 2026.