Vision-language-action model

A vision-language-action model (VLA model, or VLA) is a machine-learning model for robotics that uses visual observations and a natural-language instruction to generate actions a robot can execute. A VLA may also use proprioception, depth, touch, or other sensor data. Its defining feature is that vision and language are connected to action generation within the policy, rather than ending with a text plan that a separate, fixed controller must interpret. The term was introduced in the 2023 paper on RT-2, whose authors adapted pretrained vision-language models to represent robot actions as tokens.^[1]

VLAs are studied as a route to more general robot policies: instead of training one policy for one narrowly specified task, researchers train a model across tasks, objects, scenes, instructions, and sometimes robot bodies. The approach combines the visual and linguistic representations learned by a vision-language model with robot demonstrations that associate observations and commands with motor behavior. Evidence for broad generalization remains conditional on the training distribution, embodiment, task, and evaluation protocol. A successful demonstration or benchmark result does not establish that a model can control an unfamiliar robot or operate reliably in an unrestricted environment.^[2]

Two action-generation designs are common. Autoregressive VLAs encode actions as discrete tokens and predict them with a language-model decoder. Continuous-action VLAs attach a diffusion or flow-matching action expert to a vision-language backbone and generate a short trajectory of motor commands. Hierarchical designs further divide semantic reasoning, visuomotor control, and low-level stabilization into modules that run at different rates. These designs share the VLA label, but they differ substantially in architecture, data, latency, supported hardware, and openness.

Definition and scope

A strict operational definition requires three elements:

1. Visual input: one or more images or video frames represent the robot's surroundings. Some systems add depth, tactile measurements, or other sensor streams.
2. Language input: a natural-language instruction, goal, or subtask tells the policy what behavior is requested.
3. Action output: the model directly generates executable control commands or a trajectory, such as joint targets, end-effector pose changes, gripper commands, or whole-body actions.

This definition follows both the original RT-2 formulation and a 2025 review that excludes systems in which a vision-language component merely chooses among separately trained skills.^[1][2] There is no single universally enforced taxonomy. Authors sometimes use VLA more broadly for any robot policy that combines vision, language, and action data. It is therefore useful to state what a particular model actually predicts rather than infer its design from the label alone.

Relation to adjacent model types

A standard vision-language model maps images and text to text or another semantic representation. A VLA adds an action interface and can close the perception-action loop. Many VLAs begin from a pretrained language model or vision-language checkpoint, but a language-model decoder is not required if a separate action expert consumes the learned representation.

An embodied reasoning model can analyze a scene, estimate state, answer spatial questions, or make a plan without directly controlling a robot. PaLM-E, for example, incorporated images and sensor measurements into a large language model and could produce plans, while the demonstrated control stack delegated low-level execution to RT-1.^[5] It is an important precursor to VLAs, but its original planning configuration does not satisfy the strict direct-action definition.

A generalist robot policy may condition on language or goal images without being built from a vision-language model. Octo describes itself as a generalist robot policy. It uses a Transformer trained on robot data, supports language and goal-image task specifications, and employs a diffusion action head, but its authors distinguish it from VLA systems built by adapting a pretrained VLM.^[7] The boundary is conceptual rather than a claim that one design is more capable.

A world model predicts future states, observations, or latent dynamics. It can supply simulated training data or support planning, but predicting video or state transitions is not itself action generation. A robot system may combine a world model, a planner, and a VLA.

In 2026, NVIDIA researchers argued for going a step further and replacing the vision-language backbone itself with a video world model, producing what they call a world action model (WAM): a single network post-trained to jointly predict future video and robot actions. The February 2026 DreamZero paper reported that this design generalized to unseen tasks and environments better than the VLA baselines it tested, at the cost of a much larger and slower policy, and an August 2026 NVIDIA developer blog presented WAMs as a successor to the VLA recipe.^[31][32] Those comparisons were vendor-run, and NVIDIA continued releasing VLAs alongside them, so as of August 2026 the two designs coexisted rather than one replacing the other.

Historical development

Generalist-policy precursors

Gato, published in 2022, trained one 1.2-billion-parameter Transformer across 604 tasks that included text, image captioning, games, simulated environments, and real robot control. It serialized observations and actions into a shared token sequence and used the same weights for every task.^[3] Gato demonstrated that a single sequence model could represent heterogeneous agent behavior. It did not adapt a large pretrained VLM to robotics, so it is usually treated as a generalist-agent precursor rather than an early VLA.

RT-1, first reported in December 2022 and later published at RSS 2023, was a 35-million-parameter robot policy trained on approximately 130,000 demonstration episodes collected over 17 months with a fleet of 13 robots. The dataset covered more than 700 tasks. RT-1 combined an EfficientNet visual encoder with FiLM language conditioning, TokenLearner, and a Transformer that predicted discretized arm, base, and gripper actions. It operated at 3 Hz in the reported system.^[4] RT-1 established an influential recipe for scaling language-conditioned behavior cloning on real robots.

PaLM-E, first reported in March 2023 and published at ICML that year, connected a large language model to continuous sensor inputs by mapping images and state estimates into its embedding space. Its largest reported configuration combined a 540-billion-parameter PaLM model with a 22-billion-parameter Vision Transformer. The model addressed planning, visual question answering, and embodied reasoning. In the demonstrated mobile-manipulation system, PaLM-E issued textual plans and RT-1 executed the actions.^[5]

RT-2 and the VLA term

The RT-2 paper, first posted on 28 July 2023, introduced the term vision-language-action model. The model was co-fine-tuned on web-scale vision-language tasks and robot trajectories, with actions expressed through the language model's token vocabulary. The experiments used PaLM-E and PaLI-X backbones, including 12-billion- and 55-billion-parameter configurations.^[1]

RT-2 represented a seven-dimensional action as integers from 0 to 255. Each dimension was uniformly discretized, and 256 tokens were reserved for the resulting values. A response that looked like a short string of tokens to the decoder could therefore be interpreted as changes in position and orientation plus a gripper command. This let the same training interface handle semantic image-text examples and robot demonstrations.

The paper reported more than 6,000 robot evaluations and improvements over its baselines on several forms of generalization, including unfamiliar objects and instructions requiring semantic reasoning. Those findings belong to the paper's Google Robot setup and task distributions. The authors also noted that physical skills remained bounded by the kinds of skills present in the robot data. The largest model ran at roughly 1 to 3 Hz, while a 5-billion-parameter variant ran at about 5 Hz, illustrating the latency cost of using a large autoregressive backbone directly in the control loop.^[1]

Open and continuous-action systems

The Open X-Embodiment collaboration assembled robot datasets from 21 institutions into a shared format. The paper's current version reports more than one million trajectories, 22 robot embodiments, and 527 skills. Its RT-X experiments found positive transfer in selected cross-embodiment settings, but did not establish universal transfer to arbitrary hardware.^[6] The corpus made it possible to train generalist policies without access to a single organization's proprietary fleet.

Octo, released in 2024, trained 27-million- and 93-million-parameter policies on approximately 800,000 Open X-Embodiment trajectories. It supports language or goal-image conditioning and uses a diffusion action head. The project released model weights and training code. Because it is trained as a generalist robot policy rather than derived from an autoregressive VLM, it also illustrates why the VLA category has a disputed outer boundary.^[7]

OpenVLA, released in 2024, provided weights and code for a 7-billion-parameter VLA trained on 970,000 Open X-Embodiment episodes. Its backbone combines a Llama 2 language model with DINOv2 and SigLIP visual features. Like RT-2, it predicts discretized action tokens. In the authors' 29-task evaluation, OpenVLA achieved a 16.5-percentage-point higher average success rate than RT-2-X while using fewer parameters. This is a result under the OpenVLA study's embodiments and protocol, not a general ranking across all robots or tasks.^[8]

Physical Intelligence's π0, first reported in 2024, moved away from per-dimension action tokens. It combines a PaliGemma-based 3-billion-parameter VLM with an approximately 300-million-parameter action expert trained by flow matching. The expert generates continuous chunks of 50 actions and supports control rates up to 50 Hz in the described systems. The training mixture contained more than 10,000 hours of data across seven robot configurations and 68 tasks, together with Open X-Embodiment data.^[9]

FAST, published in 2025, offered another way to preserve autoregressive prediction while representing high-frequency action sequences more efficiently. It transforms action chunks into the frequency domain with a discrete cosine transform, quantizes the coefficients, and compresses repeated patterns with a byte-pair-encoding-style vocabulary. The FAST+ tokenizer was trained on one million robot trajectories. In experiments reported by its authors, π0-FAST matched the diffusion-based π0 on the evaluated tasks while training as much as five times faster in some settings.^[10]

π0.5 extended the π0 design with heterogeneous co-training. Its mixture included robot trajectories from other embodiments, verbal instructions, high-level semantic prediction, and web data. For mobile manipulation, the authors collected about 400 hours of data, while 97.6 percent of first-stage training examples came from sources other than the target mobile-manipulation tasks. In the paper's evaluations, the model performed multi-stage kitchen and bedroom tasks lasting about 10 to 15 minutes in homes outside its training environments. These were controlled research evaluations, not evidence of unrestricted household reliability.^[11]

OpenVLA-OFT showed that the original OpenVLA could be adapted without replacing its entire backbone. The method added action chunking, parallel decoding, continuous action representations, and an L1 regression objective. The authors reported an increase from 76.5 to 97.1 percent average success on their LIBERO setup and a 26-fold improvement in action-generation throughput relative to their OpenVLA baseline. On an ALOHA setup, the adapted policy ran at 25 Hz. These numbers compare implementations within that study and should not be combined with results from different benchmark versions or robot cells.^[12]

Architecture

Inputs and state representation

The minimal VLA input is an image plus a language instruction. Practical systems often provide multiple camera views, a history of recent frames, and proprioceptive state such as joint angles, velocities, end-effector poses, or gripper state. Tactile signals and force measurements are less common because large aligned datasets for those modalities are limited.

Images are converted to visual tokens by a vision encoder. Instructions are tokenized by a language model. Proprioceptive state is usually normalized per robot and projected into the model's hidden dimension. A Transformer can process these inputs jointly, or the VLM can produce contextual features consumed by a separate action module.

Observation history matters because a single image does not reveal velocity, contact evolution, or whether an earlier action succeeded. Longer histories increase memory and computation, so some policies use only a small frame window and repeatedly replan.

Discrete action tokens

Autoregressive VLAs reduce control to next-token prediction. For a robot with an action vector

$$\mathbf{a}_t = (a_t^1, a_t^2, \ldots, a_t^d),$$

each component can be clipped to a range, divided into bins, and assigned a token. The decoder predicts a sequence representing the full action. RT-2 and OpenVLA use 256 bins per dimension.^[1][8]

This design reuses the language-model vocabulary and loss function, making co-training with image-text examples straightforward. It also has limitations. Independent quantization can discard small changes, and autoregressive decoding adds latency because action dimensions or chunk tokens are generated sequentially. The FAST approach reduces sequence length by exploiting temporal correlation within action chunks.^[10]

Diffusion and flow-matching action experts

A continuous-action expert predicts a distribution over action chunks rather than selecting bins. Diffusion models generate an action sequence by starting with noise and applying learned denoising updates conditioned on the observation. Diffusion Policy established this approach for visuomotor behavior before it became common in VLA systems. It introduced receding-horizon action prediction, visual conditioning, and time-series diffusion networks, and evaluated the method across simulated and real manipulation tasks.^[29]

Flow matching trains a vector field that transports a simple distribution toward the action distribution. π0 uses a dedicated action expert for this purpose.^[9] Diffusion and flow matching are related continuous generative approaches, but they are not synonyms. Both can represent multimodal action distributions and produce temporally coherent chunks.

An action chunk contains several future commands:

$$\mathbf{A}_t = (\mathbf{a}_t,\mathbf{a}_{t+1},\ldots,\mathbf{a}_{t+H-1}).$$

The controller can execute all of the chunk or only its initial portion before requesting a new prediction. Executing more steps lowers inference demand but reduces responsiveness to new observations. Replanning sooner improves feedback at the cost of more compute and possible discontinuities between chunks.

Hierarchical and dual-system control

Hierarchical VLAs separate semantic processing from fast motor generation. NVIDIA's GR00T N1 uses a 1.34-billion-parameter Eagle-2 VLM as a slower reasoning module and a diffusion Transformer as an action module. The full model has about 2.2 billion parameters. The paper reports 10 Hz operation for the VLM component on an NVIDIA L40 and a 63.9-millisecond sampling time for a 16-action chunk in one bfloat16 configuration.^[13]

Figure's original Helix system uses a 7-billion-parameter VLM called System 2 at a company-reported 7 to 9 Hz and an 80-million-parameter System 1 controller at 200 Hz. System 1 generates continuous targets for 35 upper-body degrees of freedom. Figure states that the model was trained with about 500 hours of teleoperation data whose language labels were generated automatically. These details come from the company's technical announcement rather than a peer-reviewed model report.^[14]

Google's first Gemini Robotics model similarly combines a Gemini 2.0-derived VLA backbone with a local action decoder. The technical report describes backbone latency below 160 milliseconds, roughly 250 milliseconds from a raw observation to an action chunk, and effective 50 Hz execution through chunked actions. The parameter count was not disclosed. Training included thousands of hours of ALOHA 2 demonstrations collected over 12 months, alongside multimodal and embodied-reasoning data.^[15]

The hierarchy can extend below the action policy. Helix 02, announced by Figure in January 2026, adds a 10-million-parameter System 0 controller trained in simulation and on more than 1,000 hours of retargeted human motion data. Figure says System 1 produces full-body joint targets at 200 Hz and System 0 executes actuator commands at 1 kHz. The company demonstrated a four-minute dishwasher task and other dexterous behaviors, but the announcement did not provide an independent evaluation protocol or public weights.^[21]

Training

Pretraining and co-training

VLA training commonly begins with a VLM checkpoint whose visual grounding and language representations were learned from non-robot data. Robot demonstrations then add action supervision. If a model is trained only on robot trajectories, the VLM can forget parts of its original language or visual capability. Co-training retains a mixture of image-text, question-answering, or embodied-reasoning examples while optimizing the policy.

Robot data is usually learned through behavior cloning: the model predicts the action chosen by a demonstrator from the recorded observations and instruction. Discrete-action policies use cross-entropy over action tokens. Continuous policies use regression, diffusion, or flow-matching objectives. The demonstration distribution determines which physical skills the model observes, so semantic knowledge from web pretraining does not create unseen motor competence by itself.^[1]

The action labels may describe joint positions, joint changes, end-effector poses, velocities, gripper commands, or whole-body targets. Their coordinate frames, units, dimensionality, control rates, and safety limits differ across robots. Dataset mixtures therefore require normalization and embodiment-specific mappings. A common representation is an end-effector delta plus a gripper command, but it discards some morphological detail and is not suitable for every platform.

Cross-embodiment data

Open X-Embodiment standardized heterogeneous datasets while preserving information about their original robot and action spaces.^[6] Cross-embodiment training can increase task and scene diversity, but it does not remove the need to map predictions back to each robot. Body geometry, camera placement, grippers, joint limits, control frequency, and sensor calibration remain sources of mismatch.

GR00T N1 combines real robot trajectories, simulation or synthetic trajectories, and human videos. Because ordinary human video lacks robot action labels, its training pipeline uses learned latent-action or inverse-dynamics annotations for some data.^[13] Gemini Robotics 1.5 introduced a Motion Transfer mechanism intended to learn from ALOHA, bi-arm Franka, and Apptronik Apollo data without manually aligning their action spaces. The paper reports control of those three embodiments and cross-robot skill transfer within its evaluations.^[19]

NVIDIA's N1.7 documentation describes a further attempt to use human video at scale. It reports pretraining with 20,854 hours of egocentric video through the EgoScale project, followed by robot-specific fine-tuning. Its 3-billion-parameter Action Cascade uses a Cosmos-Reason2-2B vision-language backbone and a 32-layer diffusion Transformer. Claims about the resulting scaling behavior and dexterity come from NVIDIA's release material and had not been independently replicated by the research cutoff.^[22]

Adaptation

A pretrained VLA generally needs fine-tuning for a new robot, sensor layout, or task distribution. Full fine-tuning updates all weights but is expensive. Alternatives include low-rank adaptation, freezing the VLM while training an action head, or updating only selected layers.

Gemini Robotics On-Device, announced in June 2025, was optimized for local inference and initially offered through a trusted-tester program. Google reported that the model could be adapted to selected tasks with 50 to 100 demonstrations and showed adaptations from its ALOHA training embodiment to bi-arm Franka and Apollo robots.^[17] The company did not publish a parameter count or open checkpoint, so its compute requirements cannot be compared directly with open models.

SmolVLA provides an open efficiency-focused example. It has 450 million parameters, including an approximately 100-million-parameter action expert, and uses only the first half of its SmolVLM-2 decoder layers for action conditioning. The paper's pretraining set contains 22,900 episodes from 481 public community datasets. Its asynchronous inference stack separates chunk prediction from execution. The model can be trained on one GPU and deployed on consumer GPUs or CPUs, according to the authors, although achievable control rates depend on the hardware and task.^[16]

Datasets

Robot demonstrations are expensive because they require physical hardware, safe operation, reset procedures, and consistent labeling. Public datasets vary in task definition, camera layout, action representation, and data quality. Dataset size alone is therefore not a sufficient description.

BridgeData V2 contains 50,365 human demonstrations and 9,731 trajectories from a randomized pick-and-place policy, totaling 60,096 trajectories. It covers 24 environments and includes natural-language labels for all data.^[23] The published proceedings PDF supports this total; its PMLR landing-page abstract instead displays 53,896, which is inconsistent with the paper's abstract, figure, and data breakdown.

DROID uses a standardized Franka setup distributed across institutions. The RSS 2024 proceedings PDF and current project record report 76,000 trajectories, about 350 hours, 564 scenes, 86 tasks, and 52 buildings, with three synchronized RGB camera streams, camera calibration, depth information, and natural-language instructions.^[24][30] The RSS proceedings landing-page abstract instead says 65,000 trajectories, so the figure should be attributed to a specific source version.^[30] The project changelog separately reports improved camera calibrations for 36,000 episodes in April 2025 and updated language annotations for 75,000 successful episodes in December 2024. Those figures describe updated artifacts or subsets, not alternate corpus totals.^[24]

Public aggregation increases access, but dataset composition can introduce imbalance. A model may see many pick-and-place episodes and few examples of contact-rich or long-horizon behavior. Instructions can be templated, vague, or automatically generated. Failed attempts are often underrepresented even though recovery requires them. Some datasets mix control frequencies and action conventions that are difficult to normalize. A 2026 data-centric survey identifies embodiment alignment, long-horizon evaluation, multimodal supervision, and physically faithful data generation as continuing infrastructure problems.^[27]

Synthetic and simulated data can expand task and scene coverage and produce exact state labels, but simulation cannot reproduce every contact, material, sensor artifact, or actuator behavior. Human video is abundant but normally lacks robot actions and differs from robot embodiment. Both sources need validation on the target system.

Representative systems

The following table summarizes selected systems that established distinct architecture or deployment patterns. Parameter counts and rates are those disclosed by the cited authors or organizations. A blank entry means the source did not disclose a directly comparable value.

Release labels should not be read as a chronological leaderboard. The systems use different bodies, tasks, sensors, action spaces, and success criteria.

Developments from 2025 to 2026

GR00T N1.5, published by NVIDIA Research in June 2025, retained the Eagle-based VLM and diffusion Transformer pattern while updating visual grounding and adding a future-latent alignment objective. Its pretraining mixture included internal GR-1 data, Open X-Embodiment, simulation, DreamGen trajectories, and AgiBot data. NVIDIA's benchmark and robot results compare N1.5 with its own N1 configurations.^[18]

Gemini Robotics 1.5 added language reasoning traces before or between actions. Its Motion Transfer training covers multiple embodiments, and the full agentic configuration uses a separate embodied-reasoning model as an orchestrator for longer tasks. The technical report states that more than 90 percent of evaluation episodes during model development were run in simulation, while final comparisons included interleaved real-robot tests. That disclosure is important because simulation can guide iteration without replacing real-world verification.^[19]

GR00T N1.6, announced in December 2025, replaced the N1.5 vision-language backbone with an internal NVIDIA Cosmos 2B variant, doubled the diffusion Transformer from 16 to 32 layers, and removed a four-layer adapter. NVIDIA reports several thousand additional hours of teleoperated and simulated data involving YAM arms, AgiBot Genie-1, simulated Galaxea R1 Pro, and Unitree G1. The release also states that out-of-distribution language following remains challenging, even after the reported improvements.^[20]

The 2026 Helix 02 and GR00T N1.7 releases emphasize different scaling paths. Helix 02 adds a fast whole-body controller and more sensor modalities, including palm cameras and tactile sensors, within one proprietary humanoid stack.^[21] GR00T N1.7 uses human egocentric video to pretrain an open reasoning VLA intended for fine-tuning across humanoid embodiments.^[22] Neither announcement establishes general-purpose humanoid autonomy outside its disclosed tasks.

Evaluation

What is measured

The most common metric is task success rate, the fraction of episodes that satisfy a predefined goal. Studies may also report partial progress, completion time, intervention rate, path length, action smoothness, collision count, force limits, inference latency, and memory use. A success metric is meaningful only with a task definition, initial-state distribution, number of trials, and criterion for judging completion.

Generalization can refer to different changes:

• Object generalization: unfamiliar object instances or categories.
• Scene generalization: new backgrounds, layouts, lighting, or camera viewpoints.
• Instruction generalization: new phrasing or semantically related commands.
• Skill generalization: a requested behavior absent from the robot demonstrations.
• Embodiment generalization: transfer to a robot with different morphology, sensors, or action space.

These are not interchangeable. A model that handles a new color or prompt wording has not necessarily learned a new motor skill.

Benchmark landscape

By 2026, VLA results were reported on two broad kinds of benchmark. Simulation suites are cheap and repeatable and are the most widely reported; in March 2026 alone, 79 arXiv papers reported results on LIBERO.^[54] They range from older fixed-task suites such as CALVIN, LIBERO and SimplerEnv to perturbed versions of LIBERO and larger 2025-2026 suites that try to separate training conditions from test conditions. Shared real-robot benchmarks for generalist policies, which appeared from 2025 onward, run policies on physical robots operated by a third party or by a network of evaluators. Coverage across benchmarks is thin: in the catalogue compiled for the vla-eval harness paper, 81% of more than 509 models had been evaluated on only one benchmark and 6% on three or more.^[52]

"First released" gives the month of the first paper, preprint or launch announcement.

LIBERO

LIBERO was introduced as a lifelong robot-learning benchmark with 130 language-conditioned manipulation tasks and human teleoperation demonstrations. The original four suites are LIBERO-Spatial with 10 tasks, LIBERO-Object with 10, LIBERO-Goal with 10, and LIBERO-100 with 100. LIBERO-100 is divided into LIBERO-90 for pretraining and a 10-task downstream set that later VLA literature commonly calls LIBERO-Long or LIBERO-10.^[25]

LIBERO is reproducible and inexpensive relative to physical evaluation, but scores depend on the dataset conversion, observation setup, action normalization, rollout horizon, number of checkpoints, and aggregation rule. A result from one implementation should not be placed beside another without checking those details.

By 2025 and 2026, standard LIBERO scores were close to the ceiling. The LIBERO-Pro2.0 authors describe "increasingly many systems approaching saturated performance under its standard protocol",^[43] and the vla-eval harness reported in July 2026 that four checkpoints served through its LeRobot bridge reproduced their published LIBERO scores, including 100% for π0.5 and 99% for GR00T N1.7.^[56] Several follow-up benchmarks perturb LIBERO to test whether such scores reflect generalization:

• LIBERO-PRO (October 2025) varies the manipulated objects, initial states, task instructions and environments. Its authors report that models with more than 90% accuracy under standard LIBERO evaluation fell to 0.0% under their generalized setting. They attribute the gap to rote memorization of action sequences and layouts: models kept grasping when the target object was replaced, and their outputs did not change for corrupted instructions.^[41]
• LIBERO-Plus (October 2025) contains 10,030 test-only tasks across seven perturbation dimensions. It found success falling from 95% to below 30% under modest changes to camera viewpoint or robot initial state, while models were largely insensitive to changes in language. With the instruction replaced by an empty value, OpenVLA-OFT's performance on the Object suite remained largely unchanged.^[42]
• LIBERO-Pro2.0 (October 2026), which shares its lead authors at Huazhong University of Science and Technology with LIBERO-PRO, defines 22 static and 20 runtime perturbations that yield 1,648 shifted task instances. Across 17 policies whose nominal success rates ranged from 63.8% to 98.5%, runtime shifts lowered success by 26.5 percentage points on average. Robustness did not follow nominal rank: FastWAM scored higher than π0.5 under the standard protocol (97.8% against 93.3%) but lower under runtime shifts (70.1% against 73.8%).^[43]

SIMPLER and real-to-sim evaluation

SIMPLER evaluates policies trained on real robot data inside purpose-built simulations of Google Robot and WidowX BridgeData V2 setups. Its authors compared real and simulated performance for several open policies over approximately 1,500 evaluation episodes. They reported strong correlations when control and visual differences were reduced, particularly under the Visual Matching protocol.^[26]

SIMPLER is intended as a scalable proxy and diagnostic tool, not a replacement for physical trials. A simulator may misrepresent contact, friction, camera artifacts, or object dynamics. Correlation between rankings in a particular set of tasks does not imply that simulated success equals real success numerically.

Several 2025-2026 simulation benchmarks report their own checks against real-robot results, often using RoboArena as the reference. PolaRiS (December 2025) turns short video scans of real scenes into interactive simulation environments with neural reconstruction, and its authors report a much stronger correlation with real-world generalist-policy performance than existing simulated benchmarks.^[55] MolmoSpaces-Bench reports a strong sim-to-real correlation on its pick task (Pearson R = 0.96, Spearman ρ = 0.98) against results from 752 RoboArena pick tasks, and weaker but positive correlations for open and close tasks measured against RoboArena and the CAP study.^[38] RoboLab's authors found that its ranking of four DROID policies (π0.5, π0-FAST, π0 and a PaliGemma baseline) matched their RoboArena order, with a Spearman ρ of 1.00 and a Pearson r of 0.68.^[39] Each of these checks covers a small set of policies.

Other simulation suites

CALVIN, introduced in 2021, tests long-horizon language-conditioned manipulation. Policies learn from about 24 hours of teleoperated "play" data in four tabletop environments, and the long-horizon evaluation asks them to complete chains of five instructions drawn from 34 tasks over 1,000 chains.^[33] Newer suites put more distance between training and test conditions:

• RoboTwin 2.0 (June 2025) is a bimanual benchmark and data generator with 50 dual-arm tasks, five robot embodiments and an object library of 731 instances in 147 categories. Domain randomization covers clutter, lighting, background, tabletop height and language.^[34] Its leaderboard fixes training data at 50 clean demonstrations for each of the 50 tasks on the Aloha-AgileX robot and reports success over 100 trials per task in clean ("easy") and randomized ("hard") scenes.^[35]
• RoboCasa365 (March 2026) extends RoboCasa to 365 household tasks in 2,500 kitchens, with more than 600 hours of human demonstrations and more than 1,600 hours of synthetic ones.^[36] Its leaderboard scores 50 target tasks in three splits: Atomic-Seen (18 tasks), Composite-Seen (16) and Composite-Unseen (16), the last held out from pretraining and evaluated zero-shot.^[37]
• MolmoSpaces, from the Allen Institute for AI (February 2026), supplies more than 230,000 indoor scenes for MuJoCo, Isaac and ManiSkill. MolmoSpaces-Bench evaluates policies zero-shot, without fine-tuning on benchmark data, and its authors identify sensitivity to prompt phrasing, initial joint positions and camera occlusion.^[38]
• RoboLab-120 (April 2026), from NVIDIA, runs 120 pick-and-place tasks in Isaac Lab, grouped into visual, procedural and relational competencies at three difficulty levels. To avoid scoring policies fitted to the simulator, its paper evaluated policies trained exclusively on the real-world DROID dataset; the public leaderboard now also lists language-model agents and planners and marks entries that used simulation data.^[39][40]
• RoboDojo's simulation track (July 2026) has 42 tasks in NVIDIA Isaac Sim across five capability dimensions. At launch the best policy averaged an 8.80% success rate, against 76.03% for expert human teleoperators.^[44]

Real-robot evidence

Real-robot tests capture hardware and environmental effects that simulation omits, but are difficult to standardize. Studies use different robots, workcells, teleoperation data, reset conditions, prompts, and human judgment. Some report hundreds or thousands of trials; company announcements may show selected demonstrations without a complete denominator.

Cross-paper rankings are therefore usually invalid. The 16.5-point OpenVLA comparison is meaningful because the authors evaluated OpenVLA and RT-2-X under one protocol.^[8] The OpenVLA-OFT LIBERO and ALOHA improvements are meaningful against the baselines implemented in that paper.^[12] Neither number establishes superiority to later systems tested elsewhere.

The lack of standard protocols is documented in recent reviews of VLA deployment and data infrastructure. They identify inconsistent task definitions, success criteria, data splits, hardware, and scene construction as obstacles to reproducible comparison.^[2][27]

Shared real-robot benchmarks

From 2025, several groups began running physical evaluations of other people's generalist robot policies, so that a result no longer depended only on each paper's own robot cell.

RoboArena, posted in June 2025, crowd-sources double-blind A/B comparisons. An evaluator at a participating lab sets up a scene and an instruction on a DROID station, runs two anonymous policies back to back, and records progress scores and a preference; a central server fits the comparisons into a ranking with an extended Bradley-Terry model.^[45] On 1 November 2026 its official leaderboard, which includes only policies with at least 100 A/B evaluations, ranked NVIDIA's DreamZero first.^[57]

RoboChallenge, built by researchers at Dexmal and Hugging Face and described in an October 2025 report, keeps the model on the participant's own machine and lets it drive the platform's robots remotely. Testers reset each scene to match a reference image, and each rollout is graded against staged progress criteria.^[46] Its Table30 benchmark was retired on 27 May 2026 in favor of Table30 V2.^[58]

RoboDojo (July 2026) pairs its simulation suite with 18 real tasks on three bimanual arms on a standardized test rig. The organizers run official evaluations, three evaluators score each filmed trial under a double-blind protocol, and verified entries must release checkpoints, code and videos and pass a check on hidden layouts.^[44]

PhAIL, from Positronic Robotics, runs a Franka FR3 station in the DROID configuration. Its May 2026 paper replaces binary success at a fixed timeout with the distribution of time-to-success, scored against a human teleoperator on the same fixture; a scheduler picks the next model at random so the operator does not know which policy is running.^[47] On 29 October 2026 its leaderboard listed eight models on two benchmarks: a four-object pick-and-place task and a DROID everyday-tasks set of 286 tasks.^[48]

Two real-robot benchmarks from 2026 also test general-purpose language models. Robocurve's RoboDojo-RC Tier 1 adapts six RoboDojo real tasks to bimanual I2RT YAM arms and scores language models that command end-effector poses through agent tools alongside VLAs, with 120 attempts per configuration.^[49][50] Artificial Analysis's AA-Robotics v0.1, launched on 14 October 2026, gives language models running in coding-agent harnesses zero-shot control of a six-axis arm on eight tabletop tasks.^[51]

Known problems with benchmark results

Self-reported numbers. Simulation scores in VLA papers are usually produced by each paper's own team. The vla-eval maintainers write that research teams end up maintaining private evaluation forks for each benchmark and that results diverge.^[56] Their leaderboard compiles published numbers: on 20 October 2026 it listed 3,578 results for 3,161 models from 2,386 reviewed papers, and it warns that evaluation protocols are not fully standardized, so scores may not be directly comparable. Its maintainers also state that it is largely maintained by AI and that errors are possible.^[53] A June 2026 audit of LIBERO, CALVIN, SimplerEnv, RoboCasa and RoboTwin 2.0 by researchers at the Toyota Technological Institute at Chicago found that only 19.8% of state-of-the-art claims on LIBERO and 19.7% on SimplerEnv were provably statistically significant, against 53.3% on RoboCasa and 73.7% on RoboTwin 2.0.^[54] Real-robot reports share the problem. PhAIL's author surveyed 13 real-robot VLA papers from 2023 to 2025 and found that the most common sample size was 10 to 20 rollouts per condition; none of the 13 reported confidence intervals or paired tests.^[47] In a test described in the RoboChallenge report, testers who were the models' own authors placed objects strategically in "sweet spots" where a task was more likely to succeed and recorded better results than other testers.^[46] Shared benchmarks handle submissions differently: RoboDojo computes official scores itself and keeps unverified closed-source results on a separate track, RoboTwin 2.0 lists only models with public code, weights and a report, and RoboCasa365 verifies submissions before listing them.^[44][35][37]

Benchmark-specific fine-tuning. RoboLab's authors note that LIBERO often uses nearly identical environments for training and evaluation, so fine-tuning on its demonstrations trivializes the test.^[39] The Toyota Technological Institute audit showed that a 0.09-billion-parameter probe with no language encoder and no large-scale robot pretraining scores at or near the best reported LIBERO result, but well below the best reported results on RoboTwin 2.0 and RoboCasa. Its authors add that because papers choose which benchmarks to report and tune their methods for them, new papers almost always claim state-of-the-art results on every benchmark they report.^[54] Real-robot leaderboards face the same choice. RoboChallenge's Table30 offered up to 1,000 demonstrations per task and allowed a separately fine-tuned model for each task. In its launch report, π0.5 averaged 43.7% success when fine-tuned per task and 17.7% when trained as one generalist on about 50 episodes per task, and the report notes that the platform cannot check whether a user runs individually tuned models where a generalist is expected.^[46] Results also depend on how a model is prepared for each leaderboard, though setting, robots and tasks differ as well. Dexmal's DM0, entered as a per-task model, ranked second of 22 on the final Table30 leaderboard with a 62.00% success rate.^[59] On RoboDojo's simulation leaderboard, where the RoboDojo team trained the DM0 baseline on the benchmark's 100 demonstrations per task, it ranked 51st with a 0.05% success rate as of the 30 October 2026 update.^[44][60]

Hardware and harness differences. Small implementation details can move scores by tens of points. The vla-eval authors found that using the wrong proprioceptive state source for X-VLA on LIBERO cut success from 97.8% to 42%, that a quaternion-conversion mismatch dropped OpenVLA-OFT on LIBERO-Long from 95% to 56%, and that GR00T scored 0% on official SimplerEnv without an end-effector pose input that exists only in an internal simulator fork.^[52] Protocols also vary within one benchmark: SimplerEnv has three robot configurations whose results are not comparable, CALVIN's ABC-to-D and ABCD-to-D splits differ, and LIBERO papers report four or five suites.^[52] Leaderboards adjust settings as well; RoboCasa365 notes that it re-evaluated GR00T N1.5 with a horizon 1.5 times longer than in the paper's reported results.^[37] Robocurve states that RoboDojo-RC changed the original setups, instructions, embodiment and rubrics, so its scores are not directly comparable with RoboDojo, and that in its September 2026 report the human graders knew which model they were scoring.^[49] Under one fixed protocol, repeat runs can be stable: RoboDojo reports a standard deviation of at most 1.3 percentage points in overall real-world success rate across three repeated rounds for each of three policies.^[44]

Saturation of easy suites. LIBERO-PRO, LIBERO-Plus and LIBERO-Pro2.0 were all built in response to near-ceiling LIBERO scores, and RoboLab's authors list "rapid performance saturation on static task sets" among the main limitations of current benchmarks.^{[41][42][43][39]} Harder 2026 suites remained far from the ceiling. As of 1 November 2026, RoboCasa365's leader averaged 64.4 with 45.2% success on unseen composite tasks, and the top-ranked policy on RoboDojo's simulation leaderboard (agent systems are listed separately) had a 34.97% success rate.^[37][60]

Weak agreement across benchmarks. Few models are tested widely enough to compare across benchmarks; in the vla-eval paper's catalogue only 0.6% of models appeared on five or more.^[52] Where models overlap, rankings can change. In LIBERO-Pro2.0, rank under the standard protocol did not predict rank under runtime shifts.^[43] On 1 November 2026, RoboArena's official leaderboard placed DreamZero first (score 1735) and pi05_droid second (1608).^[57] RoboLab-120, whose paper had found its order of four older DROID policies identical to RoboArena's, listed the DROID π0.5 policy slightly ahead of DreamZero (28.0% against 25.7% success).^[39][40] The same pattern appears inside single suites that report several dimensions. In the RoboDojo paper, Spatial Forcing performed strongly on generalization, X-VLA led on precision, and Hy-Embodied-0.5-VLA had the best long-horizon, memory and overall results.^[44]

Capabilities and evidence limits

VLA research has demonstrated several capabilities under controlled conditions:

• Language can select among many manipulation behaviors learned from a shared demonstration corpus.
• VLM pretraining can improve visual and semantic generalization relative to robot-only baselines in some evaluations.^[1][8]
• Continuous action experts can generate smooth, high-frequency chunks for bimanual and dexterous control.^[9][13]
• Multi-embodiment training can transfer some behaviors across related and distinct robots in evaluated settings.^[6][19]
• Hierarchical policies can combine slow semantic processing with faster motor control.^[14][21]
• Smaller models can run locally and reduce dependence on a network connection.^[16][17]

Each capability has a boundary. A pretrained VLM may recognize an object without knowing how the target robot can grasp it. An action expert may generate smooth motion that is directed at the wrong object. A multi-robot model can still require embodiment-specific fine-tuning and normalization. A long selected demonstration does not reveal the rate of failed attempts.

The term generalist normally means broader than a task-specific baseline within a study. It does not mean a policy has human-level physical competence or can perform every natural-language request. The published evidence reviewed here did not establish reliable control across arbitrary robots, homes, factories, and instructions.

Limitations

Data scarcity and coverage

Published robot demonstration corpora are far smaller than web-scale text and image corpora. Robot data is costly to collect, and a trajectory is tied to a body, controller, camera setup, and environment. Large mixed datasets can still contain narrow skill coverage, inconsistent labels, and few failures. Human video and simulation improve scale but add embodiment and reality gaps.^[27]

Distribution shift

Changes in lighting, clutter, viewpoint, object geometry, material, and dynamics can change policy behavior. Closed-loop execution compounds error: an early mistake moves the robot into a state absent from demonstrations, making later predictions less reliable. Recovery requires examples of imperfect approaches, failed grasps, and corrective actions, which successful demonstrations may omit.

Long-horizon behavior

Action chunks cover seconds rather than an entire household or industrial task. Longer behavior needs subtask selection, memory, progress estimation, error detection, and recovery. π0.5 and Gemini Robotics 1.5 incorporate semantic subtasks or reasoning to address this problem, but their results remain task- and platform-specific.^[11][19]

Embodiment transfer

Two robots may differ in degrees of freedom, reach, gripper geometry, proprioception, and control frequency. A normalized end-effector action space does not capture every constraint. Motion Transfer, latent-action labeling, and embodiment adapters are active approaches, but there is no solved representation that guarantees zero-shot transfer.^[13][19][27]

Latency and deployment cost

Large VLMs may be too slow or memory-intensive for embedded hardware. Action chunking, asynchronous inference, quantization, smaller backbones, and hierarchical control reduce the burden. They introduce tradeoffs between responsiveness, semantic capacity, smoothness, and compute. Reported frequency should identify whether it refers to model inference, action execution, or a low-level controller, because these can differ by two orders of magnitude.^[1][14][21]

Reproducibility and access

Open weights do not necessarily include the full pretraining data, robot hardware, calibration, or evaluation cell. Closed systems may disclose neither parameter count nor training corpus. Licensing also differs among code, model weights, and datasets. Describing a model simply as open source can obscure those distinctions.

Safety

A VLA is not a complete robot safety system. Its predictions pass through actuators that can exert force, collide, exceed a workspace, or interact with people and fragile objects. Safety depends on the complete stack, including hardware limits, low-level controllers, perception, state estimation, emergency stops, operating procedures, and task-specific validation.

The Gemini Robotics report separates traditional physical action safety from semantic action safety. It states that the VLA should interface with safety-critical lower-level controllers for constraints such as workspace bounds, collision avoidance, and force control. The authors also evaluate an embodied-reasoning model on their ASIMOV semantic-safety datasets and describe constitutional post-training. Those experiments are an initial study of one system, not a guarantee that the action policy will remain safe under all inputs.^[15]

Safety evaluation must distinguish safe failure, unsafe failure, safe success, and unsafe success. A robot can complete a task while using excessive force or entering a hazardous region. ForesightSafety-VLA, a 2026 preprint benchmark, formalizes 13 safety categories spanning physical interaction, language-side hazards, and perception-side hazards. Its simulation experiments use cumulative safety cost and risk-exposure time in addition to endpoint success. The benchmark's authors report nontrivial safety cost even for the strongest baselines they tested, particularly under structural and visual changes. These findings are limited to the benchmark's 66 base scenarios and five simulated embodiments.^[28]

Practical deployments should retain independent constraints beneath the learned policy, evaluate the complete system on the intended hardware, and monitor near-misses rather than only completed tasks. Safety claims based on selected videos or task success alone are insufficient.

Openness and deployment

Public releases for OpenVLA, Octo, the π0 family, the GR00T family, and SmolVLA provide varying combinations of weights, inference code, fine-tuning code, and datasets. LeRobot supplies a PyTorch-based ecosystem and dataset format used by SmolVLA and other policies. Availability supports reproducibility, but code, model weights, and data can have different licenses. Users must examine the license and release notes for each artifact rather than treating open source, open weight, and commercially licensed as interchangeable.

Closed systems include RT-2, Gemini Robotics, and Figure's Helix family. Their papers or technical posts can still disclose architecture and evaluations, but independent replication is limited without weights, training data, and comparable hardware.

Local deployment can reduce network latency and keep sensor data on the robot. SmolVLA and Gemini Robotics On-Device represent open and closed approaches to this goal.^[16][17] Local inference does not by itself establish real-time behavior or safety; the result depends on accelerator speed, camera pipeline, chunk length, controller rate, and the robot's dynamics.

As of 28 July 2026, VLA development remained a rapidly changing research area. The strongest evidence comes from direct comparisons under shared protocols, repeated physical trials, released code and checkpoints, and clearly documented failure cases. Company demonstrations and self-reported benchmark records can identify technical directions, but require independent reproduction before being treated as general capability.

Continue reading

• PPhysical IntelligencePhysical Intelligence (also known as Pi or π) is an American artificial intelligence robotics company that builds general-purpose foundation models and learning algorithms designed to control any robot for any…Listed in See also
• EEmbodied AIEmbodied AI is an area of artificial intelligence concerned with agents whose perception, action, learning, and reasoning are coupled through interaction with an environment.Listed in See also
• FFoundation ModelsA foundation model is a machine learning model trained on broad data, generally with self-supervision at scale, that can be adapted to a wide range of downstream tasks.Listed in See also
• Google DeepMindGoogle DeepMind is an artificial intelligence research and model-development organization at Google.Listed in See also
• Isaac GR00TIsaac GR00T (short for Generalist Robot 00 Technology) is an open foundation model family from NVIDIA that gives humanoid robots a single, pretrained "AI brain" for perception, reasoning, and motor control.Listed in See also
• RRobot learningRobot learning studies how robots acquire or improve behavior from data and experience.Listed in See also

More to read

• Robotics
• RT-2
• LIBERO
• OpenVLA
• RoboDojo
• Gemini Robotics

Topics:Embodied AIMultimodal AIRobotics

References

1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9Zitkovich, B. et al. (2023). "RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control." *Proceedings of The 7th Conference on Robot Learning*, PMLR 229:2165-2183. proceedings.mlr.press/...zitkovich23a
2. ^1 ^2 ^3Kawaharazuka, K., Oh, J., Yamada, J., Posner, I., and Zhu, Y. (2025). "Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications." arXiv:2510.07077. arxiv.org/...2510.07077
3. ^Reed, S. et al. (2022). "A Generalist Agent." *Transactions on Machine Learning Research*. openreview.net/forum
4. ^1 ^2Brohan, A. et al. (2023). "RT-1: Robotics Transformer for Real-World Control at Scale." *Robotics: Science and Systems XIX*, paper 025. roboticsproceedings.org/...p025
5. ^1 ^2Driess, D. et al. (2023). "PaLM-E: An Embodied Multimodal Language Model." *Proceedings of the 40th International Conference on Machine Learning*, PMLR 202:8469-8488. proceedings.mlr.press/...driess23a
6. ^1 ^2 ^3 ^4Open X-Embodiment Collaboration et al. (2024). "Open X-Embodiment: Robotic Learning Datasets and RT-X Models." *2024 IEEE International Conference on Robotics and Automation*. doi.org/...ICRA57147.2024.10611477
7. ^1 ^2 ^3Octo Model Team et al. (2024). "Octo: An Open-Source Generalist Robot Policy." *Robotics: Science and Systems XX*, paper 090. roboticsproceedings.org/...p090
8. ^1 ^2 ^3 ^4 ^5Kim, M. J. et al. (2025). "OpenVLA: An Open-Source Vision-Language-Action Model." *Proceedings of The 8th Conference on Robot Learning*, PMLR 270. proceedings.mlr.press/...kim25c
9. ^1 ^2 ^3 ^4 ^5Black, K. et al. (2025). "π0: A Vision-Language-Action Flow Model for General Robot Control." *Robotics: Science and Systems XXI*, paper 010. roboticsproceedings.org/...p010
10. ^1 ^2Pertsch, K. et al. (2025). "FAST: Efficient Action Tokenization for Vision-Language-Action Models." *Robotics: Science and Systems XXI*, paper 012. roboticsproceedings.org/...p012
11. ^1 ^2Black, K. et al. (2025). "π0.5: a Vision-Language-Action Model with Open-World Generalization." *Proceedings of The 9th Conference on Robot Learning*, PMLR 305:17-40. proceedings.mlr.press/...black25a
12. ^1 ^2Kim, M. J. et al. (2025). "Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success." *Robotics: Science and Systems XXI*, paper 017. roboticsproceedings.org/...p017
13. ^1 ^2 ^3 ^4 ^5Bjorck, J. et al. (2025). "GR00T N1: An Open Foundation Model for Generalist Humanoid Robots." arXiv:2503.14734. arxiv.org/...2503.14734
14. ^1 ^2 ^3 ^4 ^5Figure AI. (2025). "Helix: A Vision-Language-Action Model for Generalist Humanoid Control." figure.ai/...helix
15. ^1 ^2 ^3Gemini Robotics Team. (2025). "Gemini Robotics: Bringing AI into the Physical World." arXiv:2503.20020. arxiv.org/...2503.20020
16. ^1 ^2 ^3 ^4Shukor, M. et al. (2025). "SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics." arXiv:2506.01844. arxiv.org/...2506.01844
17. ^1 ^2 ^3Google DeepMind. (2025). "Gemini Robotics On-Device brings AI to local robotic devices." deepmind.google/...ngs-ai-to-local-robotic-devices
18. ^NVIDIA Research. (2025). "GR00T N1.5: An Improved Open Foundation Model for Generalist Humanoid Robots." research.nvidia.com/...gr00t-n1_5
19. ^1 ^2 ^3 ^4 ^5 ^6Gemini Robotics Team. (2025). "Gemini Robotics 1.5: Pushing the Frontier of Generalist Robots with Advanced Embodied Reasoning, Thinking, and Motion Transfer." arXiv:2510.03342. arxiv.org/...2510.03342
20. ^1 ^2NVIDIA Research. (2025). "GR00T N1.6: An Improved Open Foundation Model for Generalist Humanoid Robots." research.nvidia.com/...gr00t-n1_6
21. ^1 ^2 ^3 ^4 ^5Figure AI. (2026). "Introducing Helix 02: Full-Body Autonomy." figure.ai/...helix-02
22. ^1 ^2 ^3NVIDIA. (2026). "NVIDIA Isaac GR00T N1.7: Open Reasoning VLA Model for Humanoid Robots." huggingface.co/...gr00t-n1-7
23. ^1 ^2Walke, H. et al. (2023). "BridgeData V2: A Dataset for Robot Learning at Scale." *Proceedings of The 7th Conference on Robot Learning*, PMLR 229. Published proceedings PDF. proceedings.mlr.press/...walke23a.pdf
24. ^1 ^2 ^3DROID project. (updated 2025). "DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset." Current dataset and project record. droid-dataset.github.io
25. ^1 ^2Liu, B. et al. (2023). "LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning." *NeurIPS 2023 Datasets and Benchmarks Track*. proceedings.neurips.cc/...-Datasets_and_Benchmarks
26. ^1 ^2Li, X. et al. (2025). "Evaluating Real-World Robot Manipulation Policies in Simulation." *Proceedings of The 8th Conference on Robot Learning*, PMLR 270:3705-3728. proceedings.mlr.press/...li25c
27. ^1 ^2 ^3 ^4Wang, Z. et al. (2026). "Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines." arXiv:2604.23001. arxiv.org/...2604.23001
28. ^Lyu, M. et al. (2026). "ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models." arXiv:2606.27079. arxiv.org/...2606.27079
29. ^Chi, C. et al. (2025). "Diffusion Policy: Visuomotor Policy Learning via Action Diffusion." *The International Journal of Robotics Research*, 44(10-11). doi.org/...02783649241273668
30. ^1 ^2 ^3Khazatsky, A. et al. (2024). "DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset." *Robotics: Science and Systems XX*, paper 120. roboticsproceedings.org/...p120
31. ^Ye, S. et al. (2026). "World Action Models are Zero-shot Policies." arXiv:2602.15922. arxiv.org/...2602.15922
32. ^NVIDIA Technical Blog. (2026). "Beyond VLAs: How World Action Models Reshape Robot Manipulation." August 4, 2026. developer.nvidia.com/...reshape-robot-manipulation
33. ^1 ^2Mees, O., Hermann, L., Rosete-Beas, E., and Burgard, W. (2021). "CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks." arXiv:2112.03227. arxiv.org/...2112.03227
34. ^1 ^2Chen, T. et al. (2025). "RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation." arXiv:2506.18088. arxiv.org/...2506.18088
35. ^1 ^2 ^3RoboTwin 2.0 Leaderboard. Benchmark setting and submission policy. Accessed 1 November 2026. robotwin-platform.github.io/leaderboard
36. ^1 ^2Nasiriany, S., Nasiriany, S., Maddukuri, A., and Zhu, Y. (2026). "RoboCasa365: A Large-Scale Simulation Framework for Training and Benchmarking Generalist Robots." arXiv:2603.04356. arxiv.org/...2603.04356
37. ^1 ^2 ^3 ^4 ^5 ^6RoboCasa365 Leaderboard. Updated 1 November 2026. Accessed 1 November 2026. robocasa.ai/leaderboard
38. ^1 ^2 ^3Kim, Y. et al. (2026). "MolmoSpaces: A Large-Scale Open Ecosystem for Robot Navigation and Manipulation." arXiv:2602.11337. arxiv.org/...2602.11337
39. ^1 ^2 ^3 ^4 ^5 ^6Yang, J. X. et al. (2026). "RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies." arXiv:2604.09860. arxiv.org/...2604.09860
40. ^1 ^2 ^3 ^4NVIDIA. "RoboLab Leaderboard." Accessed 1 November 2026. research.nvidia.com/...leaderboard
41. ^1 ^2 ^3Zhou, X. et al. (2025). "LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization." arXiv:2510.03827. arxiv.org/...2510.03827
42. ^1 ^2 ^3Fei, S. et al. (2025). "LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models." arXiv:2510.13626. arxiv.org/...2510.13626
43. ^1 ^2 ^3 ^4Zhou, X. et al. (2026). "LIBERO-Pro2.0: A Benchmark for Evaluating the Robustness of Embodied Foundation Models." arXiv:2610.24587. arxiv.org/...2610.24587
44. ^1 ^2 ^3 ^4 ^5 ^6 ^7Chen, T., Chen, Y. et al. (2026). "RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies." arXiv:2607.04434. arxiv.org/...2607.04434
45. ^1 ^2Atreya, P., Pertsch, K., Lee, T. et al. (2025). "RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies." arXiv:2506.18123. arxiv.org/...2506.18123
46. ^1 ^2 ^3 ^4Yakefu, A. et al. (2025). "RoboChallenge: Large-scale Real-robot Evaluation of Embodied Policies." arXiv:2510.17950. arxiv.org/...2510.17950
47. ^1 ^2 ^3Arkhangelskiy, S. (2026). "PhAIL: A Real-Robot VLA Benchmark and Distributional Methodology." arXiv:2605.29710. arxiv.org/...2605.29710
48. ^Positronic Robotics. "PhAIL: Physical AI Leaderboard." Updated 29 October 2026. Accessed 1 November 2026. phail.ai
49. ^1 ^2 ^3Machcha, S., Menon, A., Zou, S., Chan, T. K., and Chooi, J. (2026). "Opus 5.5 on RoboDojo-RC Tier 1: higher scores at lower cost." Robocurve, 23 September 2026. robocurve.org/opus-5-5-robodojo-rc-tier-1
50. ^Robocurve. "Real-world evaluations of physical AI." RoboDojo-RC Tier 1 leaderboard. Accessed 1 November 2026. robocurve.org
51. ^1 ^2Artificial Analysis. (2026). "Welcome to the era of embodied AI: Introducing AA-Robotics v0.1." 14 October 2026. artificialanalysis.ai/...aa-robotics
52. ^1 ^2 ^3 ^4Choi, S. et al. (2026). "vla-eval: A Unified Evaluation Harness for Vision-Language-Action Models." arXiv:2603.13966. arxiv.org/...2603.13966
53. ^1 ^2 ^3vla-evaluation-harness. "VLA Model Leaderboard (beta)." Last updated 20 October 2026. Accessed 1 November 2026. allenai.github.io/...leaderboard
54. ^1 ^2 ^3Jiang, T., Tan, X., Wheeler, S., Sun, L., Ayalew, T. W., and Walter, M. (2026). "What Are We Actually Benchmarking in Robot Manipulation?" arXiv:2606.04233. arxiv.org/...2606.04233
55. ^Jain, A. et al. (2025). "PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policies." arXiv:2512.16881. arxiv.org/...2512.16881
56. ^1 ^2allenai/vla-evaluation-harness. GitHub repository README, "Latest News." Accessed 1 November 2026. github.com/...vla-evaluation-harness
57. ^1 ^2RoboArena. "Policy Leaderboard" (Official view). Last updated 1 November 2026. Accessed 1 November 2026. robo-arena.github.io/leaderboard
58. ^1 ^2RoboChallenge. "News: The End of Table 30, the Beginning of v2." 27 May 2026. robochallenge.ai/news
59. ^RoboChallenge. Table30 leaderboard data. Accessed 1 November 2026. robochallenge.ai/...leaderboard_all.json
60. ^1 ^2RoboDojo. "Leaderboard" (simulation leaderboard, updated 30 October 2026). Accessed 1 November 2026. robodojo-benchmark.com/leaderboard
61. ^Allen Institute for AI. "MolmoSpaces Leaderboard." Accessed 1 November 2026. molmospaces.allen.ai/leaderboard

Show all 61 references