Vision-language-action model

A vision-language-action model (VLA model, or VLA) is a machine-learning model for robotics that uses visual observations and a natural-language instruction to generate actions a robot can execute. A VLA may also use proprioception, depth, touch, or other sensor data. Its defining feature is that vision and language are connected to action generation within the policy, rather than ending with a text plan that a separate, fixed controller must interpret. The term was introduced in the 2023 paper on RT-2, whose authors adapted pretrained vision-language models to represent robot actions as tokens.^[1]

VLAs are studied as a route to more general robot policies: instead of training one policy for one narrowly specified task, researchers train a model across tasks, objects, scenes, instructions, and sometimes robot bodies. The approach combines the visual and linguistic representations learned by a vision-language model with robot demonstrations that associate observations and commands with motor behavior. Evidence for broad generalization remains conditional on the training distribution, embodiment, task, and evaluation protocol. A successful demonstration or benchmark result does not establish that a model can control an unfamiliar robot or operate reliably in an unrestricted environment.^[2]

Two action-generation designs are common. Autoregressive VLAs encode actions as discrete tokens and predict them with a language-model decoder. Continuous-action VLAs attach a diffusion or flow-matching action expert to a vision-language backbone and generate a short trajectory of motor commands. Hierarchical designs further divide semantic reasoning, visuomotor control, and low-level stabilization into modules that run at different rates. These designs share the VLA label, but they differ substantially in architecture, data, latency, supported hardware, and openness.

Definition and scope

A strict operational definition requires three elements:

1. Visual input: one or more images or video frames represent the robot's surroundings. Some systems add depth, tactile measurements, or other sensor streams.
2. Language input: a natural-language instruction, goal, or subtask tells the policy what behavior is requested.
3. Action output: the model directly generates executable control commands or a trajectory, such as joint targets, end-effector pose changes, gripper commands, or whole-body actions.

This definition follows both the original RT-2 formulation and a 2025 review that excludes systems in which a vision-language component merely chooses among separately trained skills.^[1][2] There is no single universally enforced taxonomy. Authors sometimes use VLA more broadly for any robot policy that combines vision, language, and action data. It is therefore useful to state what a particular model actually predicts rather than infer its design from the label alone.

Relation to adjacent model types

A standard vision-language model maps images and text to text or another semantic representation. A VLA adds an action interface and can close the perception-action loop. Many VLAs begin from a pretrained language model or vision-language checkpoint, but a language-model decoder is not required if a separate action expert consumes the learned representation.

An embodied reasoning model can analyze a scene, estimate state, answer spatial questions, or make a plan without directly controlling a robot. PaLM-E, for example, incorporated images and sensor measurements into a large language model and could produce plans, while the demonstrated control stack delegated low-level execution to RT-1.^[5] It is an important precursor to VLAs, but its original planning configuration does not satisfy the strict direct-action definition.

A generalist robot policy may condition on language or goal images without being built from a vision-language model. Octo describes itself as a generalist robot policy. It uses a Transformer trained on robot data, supports language and goal-image task specifications, and employs a diffusion action head, but its authors distinguish it from VLA systems built by adapting a pretrained VLM.^[7] The boundary is conceptual rather than a claim that one design is more capable.

A world model predicts future states, observations, or latent dynamics. It can supply simulated training data or support planning, but predicting video or state transitions is not itself action generation. A robot system may combine a world model, a planner, and a VLA.

Historical development

Generalist-policy precursors

Gato, published in 2022, trained one 1.2-billion-parameter Transformer across 604 tasks that included text, image captioning, games, simulated environments, and real robot control. It serialized observations and actions into a shared token sequence and used the same weights for every task.^[3] Gato demonstrated that a single sequence model could represent heterogeneous agent behavior. It did not adapt a large pretrained VLM to robotics, so it is usually treated as a generalist-agent precursor rather than an early VLA.

RT-1, first reported in December 2022 and later published at RSS 2023, was a 35-million-parameter robot policy trained on approximately 130,000 demonstration episodes collected over 17 months with a fleet of 13 robots. The dataset covered more than 700 tasks. RT-1 combined an EfficientNet visual encoder with FiLM language conditioning, TokenLearner, and a Transformer that predicted discretized arm, base, and gripper actions. It operated at 3 Hz in the reported system.^[4] RT-1 established an influential recipe for scaling language-conditioned behavior cloning on real robots.

PaLM-E, first reported in March 2023 and published at ICML that year, connected a large language model to continuous sensor inputs by mapping images and state estimates into its embedding space. Its largest reported configuration combined a 540-billion-parameter PaLM model with a 22-billion-parameter Vision Transformer. The model addressed planning, visual question answering, and embodied reasoning. In the demonstrated mobile-manipulation system, PaLM-E issued textual plans and RT-1 executed the actions.^[5]

RT-2 and the VLA term

The RT-2 paper, first posted on 28 July 2023, introduced the term vision-language-action model. The model was co-fine-tuned on web-scale vision-language tasks and robot trajectories, with actions expressed through the language model's token vocabulary. The experiments used PaLM-E and PaLI-X backbones, including 12-billion- and 55-billion-parameter configurations.^[1]

RT-2 represented a seven-dimensional action as integers from 0 to 255. Each dimension was uniformly discretized, and 256 tokens were reserved for the resulting values. A response that looked like a short string of tokens to the decoder could therefore be interpreted as changes in position and orientation plus a gripper command. This let the same training interface handle semantic image-text examples and robot demonstrations.

The paper reported more than 6,000 robot evaluations and improvements over its baselines on several forms of generalization, including unfamiliar objects and instructions requiring semantic reasoning. Those findings belong to the paper's Google Robot setup and task distributions. The authors also noted that physical skills remained bounded by the kinds of skills present in the robot data. The largest model ran at roughly 1 to 3 Hz, while a 5-billion-parameter variant ran at about 5 Hz, illustrating the latency cost of using a large autoregressive backbone directly in the control loop.^[1]

Open and continuous-action systems

The Open X-Embodiment collaboration assembled robot datasets from 21 institutions into a shared format. The paper's current version reports more than one million trajectories, 22 robot embodiments, and 527 skills. Its RT-X experiments found positive transfer in selected cross-embodiment settings, but did not establish universal transfer to arbitrary hardware.^[6] The corpus made it possible to train generalist policies without access to a single organization's proprietary fleet.

Octo, released in 2024, trained 27-million- and 93-million-parameter policies on approximately 800,000 Open X-Embodiment trajectories. It supports language or goal-image conditioning and uses a diffusion action head. The project released model weights and training code. Because it is trained as a generalist robot policy rather than derived from an autoregressive VLM, it also illustrates why the VLA category has a disputed outer boundary.^[7]

OpenVLA, released in 2024, provided weights and code for a 7-billion-parameter VLA trained on 970,000 Open X-Embodiment episodes. Its backbone combines a Llama 2 language model with DINOv2 and SigLIP visual features. Like RT-2, it predicts discretized action tokens. In the authors' 29-task evaluation, OpenVLA achieved a 16.5-percentage-point higher average success rate than RT-2-X while using fewer parameters. This is a result under the OpenVLA study's embodiments and protocol, not a general ranking across all robots or tasks.^[8]

Physical Intelligence's π0, first reported in 2024, moved away from per-dimension action tokens. It combines a PaliGemma-based 3-billion-parameter VLM with an approximately 300-million-parameter action expert trained by flow matching. The expert generates continuous chunks of 50 actions and supports control rates up to 50 Hz in the described systems. The training mixture contained more than 10,000 hours of data across seven robot configurations and 68 tasks, together with Open X-Embodiment data.^[9]

FAST, published in 2025, offered another way to preserve autoregressive prediction while representing high-frequency action sequences more efficiently. It transforms action chunks into the frequency domain with a discrete cosine transform, quantizes the coefficients, and compresses repeated patterns with a byte-pair-encoding-style vocabulary. The FAST+ tokenizer was trained on one million robot trajectories. In experiments reported by its authors, π0-FAST matched the diffusion-based π0 on the evaluated tasks while training as much as five times faster in some settings.^[10]

π0.5 extended the π0 design with heterogeneous co-training. Its mixture included robot trajectories from other embodiments, verbal instructions, high-level semantic prediction, and web data. For mobile manipulation, the authors collected about 400 hours of data, while 97.6 percent of first-stage training examples came from sources other than the target mobile-manipulation tasks. In the paper's evaluations, the model performed multi-stage kitchen and bedroom tasks lasting about 10 to 15 minutes in homes outside its training environments. These were controlled research evaluations, not evidence of unrestricted household reliability.^[11]

OpenVLA-OFT showed that the original OpenVLA could be adapted without replacing its entire backbone. The method added action chunking, parallel decoding, continuous action representations, and an L1 regression objective. The authors reported an increase from 76.5 to 97.1 percent average success on their LIBERO setup and a 26-fold improvement in action-generation throughput relative to their OpenVLA baseline. On an ALOHA setup, the adapted policy ran at 25 Hz. These numbers compare implementations within that study and should not be combined with results from different benchmark versions or robot cells.^[12]

Architecture

Inputs and state representation

The minimal VLA input is an image plus a language instruction. Practical systems often provide multiple camera views, a history of recent frames, and proprioceptive state such as joint angles, velocities, end-effector poses, or gripper state. Tactile signals and force measurements are less common because large aligned datasets for those modalities are limited.

Images are converted to visual tokens by a vision encoder. Instructions are tokenized by a language model. Proprioceptive state is usually normalized per robot and projected into the model's hidden dimension. A Transformer can process these inputs jointly, or the VLM can produce contextual features consumed by a separate action module.

Observation history matters because a single image does not reveal velocity, contact evolution, or whether an earlier action succeeded. Longer histories increase memory and computation, so some policies use only a small frame window and repeatedly replan.

Discrete action tokens

Autoregressive VLAs reduce control to next-token prediction. For a robot with an action vector

$$\mathbf{a}_t = (a_t^1, a_t^2, \ldots, a_t^d),$$

each component can be clipped to a range, divided into bins, and assigned a token. The decoder predicts a sequence representing the full action. RT-2 and OpenVLA use 256 bins per dimension.^[1][8]

This design reuses the language-model vocabulary and loss function, making co-training with image-text examples straightforward. It also has limitations. Independent quantization can discard small changes, and autoregressive decoding adds latency because action dimensions or chunk tokens are generated sequentially. The FAST approach reduces sequence length by exploiting temporal correlation within action chunks.^[10]

Diffusion and flow-matching action experts

A continuous-action expert predicts a distribution over action chunks rather than selecting bins. Diffusion models generate an action sequence by starting with noise and applying learned denoising updates conditioned on the observation. Diffusion Policy established this approach for visuomotor behavior before it became common in VLA systems. It introduced receding-horizon action prediction, visual conditioning, and time-series diffusion networks, and evaluated the method across simulated and real manipulation tasks.^[29]

Flow matching trains a vector field that transports a simple distribution toward the action distribution. π0 uses a dedicated action expert for this purpose.^[9] Diffusion and flow matching are related continuous generative approaches, but they are not synonyms. Both can represent multimodal action distributions and produce temporally coherent chunks.

An action chunk contains several future commands:

$$\mathbf{A}_t = (\mathbf{a}_t,\mathbf{a}_{t+1},\ldots,\mathbf{a}_{t+H-1}).$$

The controller can execute all of the chunk or only its initial portion before requesting a new prediction. Executing more steps lowers inference demand but reduces responsiveness to new observations. Replanning sooner improves feedback at the cost of more compute and possible discontinuities between chunks.

Hierarchical and dual-system control

Hierarchical VLAs separate semantic processing from fast motor generation. NVIDIA's GR00T N1 uses a 1.34-billion-parameter Eagle-2 VLM as a slower reasoning module and a diffusion Transformer as an action module. The full model has about 2.2 billion parameters. The paper reports 10 Hz operation for the VLM component on an NVIDIA L40 and a 63.9-millisecond sampling time for a 16-action chunk in one bfloat16 configuration.^[13]

Figure's original Helix system uses a 7-billion-parameter VLM called System 2 at a company-reported 7 to 9 Hz and an 80-million-parameter System 1 controller at 200 Hz. System 1 generates continuous targets for 35 upper-body degrees of freedom. Figure states that the model was trained with about 500 hours of teleoperation data whose language labels were generated automatically. These details come from the company's technical announcement rather than a peer-reviewed model report.^[14]

Google's first Gemini Robotics model similarly combines a Gemini 2.0-derived VLA backbone with a local action decoder. The technical report describes backbone latency below 160 milliseconds, roughly 250 milliseconds from a raw observation to an action chunk, and effective 50 Hz execution through chunked actions. The parameter count was not disclosed. Training included thousands of hours of ALOHA 2 demonstrations collected over 12 months, alongside multimodal and embodied-reasoning data.^[15]

The hierarchy can extend below the action policy. Helix 02, announced by Figure in January 2026, adds a 10-million-parameter System 0 controller trained in simulation and on more than 1,000 hours of retargeted human motion data. Figure says System 1 produces full-body joint targets at 200 Hz and System 0 executes actuator commands at 1 kHz. The company demonstrated a four-minute dishwasher task and other dexterous behaviors, but the announcement did not provide an independent evaluation protocol or public weights.^[21]

Training

Pretraining and co-training

VLA training commonly begins with a VLM checkpoint whose visual grounding and language representations were learned from non-robot data. Robot demonstrations then add action supervision. If a model is trained only on robot trajectories, the VLM can forget parts of its original language or visual capability. Co-training retains a mixture of image-text, question-answering, or embodied-reasoning examples while optimizing the policy.

Robot data is usually learned through behavior cloning: the model predicts the action chosen by a demonstrator from the recorded observations and instruction. Discrete-action policies use cross-entropy over action tokens. Continuous policies use regression, diffusion, or flow-matching objectives. The demonstration distribution determines which physical skills the model observes, so semantic knowledge from web pretraining does not create unseen motor competence by itself.^[1]

The action labels may describe joint positions, joint changes, end-effector poses, velocities, gripper commands, or whole-body targets. Their coordinate frames, units, dimensionality, control rates, and safety limits differ across robots. Dataset mixtures therefore require normalization and embodiment-specific mappings. A common representation is an end-effector delta plus a gripper command, but it discards some morphological detail and is not suitable for every platform.

Cross-embodiment data

Open X-Embodiment standardized heterogeneous datasets while preserving information about their original robot and action spaces.^[6] Cross-embodiment training can increase task and scene diversity, but it does not remove the need to map predictions back to each robot. Body geometry, camera placement, grippers, joint limits, control frequency, and sensor calibration remain sources of mismatch.

GR00T N1 combines real robot trajectories, simulation or synthetic trajectories, and human videos. Because ordinary human video lacks robot action labels, its training pipeline uses learned latent-action or inverse-dynamics annotations for some data.^[13] Gemini Robotics 1.5 introduced a Motion Transfer mechanism intended to learn from ALOHA, bi-arm Franka, and Apptronik Apollo data without manually aligning their action spaces. The paper reports control of those three embodiments and cross-robot skill transfer within its evaluations.^[19]

NVIDIA's N1.7 documentation describes a further attempt to use human video at scale. It reports pretraining with 20,854 hours of egocentric video through the EgoScale project, followed by robot-specific fine-tuning. Its 3-billion-parameter Action Cascade uses a Cosmos-Reason2-2B vision-language backbone and a 32-layer diffusion Transformer. Claims about the resulting scaling behavior and dexterity come from NVIDIA's release material and had not been independently replicated by the research cutoff.^[22]

Adaptation

A pretrained VLA generally needs fine-tuning for a new robot, sensor layout, or task distribution. Full fine-tuning updates all weights but is expensive. Alternatives include low-rank adaptation, freezing the VLM while training an action head, or updating only selected layers.

Gemini Robotics On-Device, announced in June 2025, was optimized for local inference and initially offered through a trusted-tester program. Google reported that the model could be adapted to selected tasks with 50 to 100 demonstrations and showed adaptations from its ALOHA training embodiment to bi-arm Franka and Apollo robots.^[17] The company did not publish a parameter count or open checkpoint, so its compute requirements cannot be compared directly with open models.

SmolVLA provides an open efficiency-focused example. It has 450 million parameters, including an approximately 100-million-parameter action expert, and uses only the first half of its SmolVLM-2 decoder layers for action conditioning. The paper's pretraining set contains 22,900 episodes from 481 public community datasets. Its asynchronous inference stack separates chunk prediction from execution. The model can be trained on one GPU and deployed on consumer GPUs or CPUs, according to the authors, although achievable control rates depend on the hardware and task.^[16]

Datasets

Robot demonstrations are expensive because they require physical hardware, safe operation, reset procedures, and consistent labeling. Public datasets vary in task definition, camera layout, action representation, and data quality. Dataset size alone is therefore not a sufficient description.

Scroll sideways for more →

BridgeData V2 contains 50,365 human demonstrations and 9,731 trajectories from a randomized pick-and-place policy, totaling 60,096 trajectories. It covers 24 environments and includes natural-language labels for all data.^[23] The published proceedings PDF supports this total; its PMLR landing-page abstract instead displays 53,896, which is inconsistent with the paper's abstract, figure, and data breakdown.

DROID uses a standardized Franka setup distributed across institutions. The RSS 2024 proceedings PDF and current project record report 76,000 trajectories, about 350 hours, 564 scenes, 86 tasks, and 52 buildings, with three synchronized RGB camera streams, camera calibration, depth information, and natural-language instructions.^[24][30] The RSS proceedings landing-page abstract instead says 65,000 trajectories, so the figure should be attributed to a specific source version.^[30] The project changelog separately reports improved camera calibrations for 36,000 episodes in April 2025 and updated language annotations for 75,000 successful episodes in December 2024. Those figures describe updated artifacts or subsets, not alternate corpus totals.^[24]

Public aggregation increases access, but dataset composition can introduce imbalance. A model may see many pick-and-place episodes and few examples of contact-rich or long-horizon behavior. Instructions can be templated, vague, or automatically generated. Failed attempts are often underrepresented even though recovery requires them. Some datasets mix control frequencies and action conventions that are difficult to normalize. A 2026 data-centric survey identifies embodiment alignment, long-horizon evaluation, multimodal supervision, and physically faithful data generation as continuing infrastructure problems.^[27]

Synthetic and simulated data can expand task and scene coverage and produce exact state labels, but simulation cannot reproduce every contact, material, sensor artifact, or actuator behavior. Human video is abundant but normally lacks robot actions and differs from robot embodiment. Both sources need validation on the target system.

Representative systems

The following table summarizes selected systems that established distinct architecture or deployment patterns. Parameter counts and rates are those disclosed by the cited authors or organizations. A blank entry means the source did not disclose a directly comparable value.

Scroll sideways for more →

Release labels should not be read as a chronological leaderboard. The systems use different bodies, tasks, sensors, action spaces, and success criteria.

Developments from 2025 to 2026

GR00T N1.5, published by NVIDIA Research in June 2025, retained the Eagle-based VLM and diffusion Transformer pattern while updating visual grounding and adding a future-latent alignment objective. Its pretraining mixture included internal GR-1 data, Open X-Embodiment, simulation, DreamGen trajectories, and AgiBot data. NVIDIA's benchmark and robot results compare N1.5 with its own N1 configurations.^[18]

Gemini Robotics 1.5 added language reasoning traces before or between actions. Its Motion Transfer training covers multiple embodiments, and the full agentic configuration uses a separate embodied-reasoning model as an orchestrator for longer tasks. The technical report states that more than 90 percent of evaluation episodes during model development were run in simulation, while final comparisons included interleaved real-robot tests. That disclosure is important because simulation can guide iteration without replacing real-world verification.^[19]

GR00T N1.6, announced in December 2025, replaced the N1.5 vision-language backbone with an internal NVIDIA Cosmos 2B variant, doubled the diffusion Transformer from 16 to 32 layers, and removed a four-layer adapter. NVIDIA reports several thousand additional hours of teleoperated and simulated data involving YAM arms, AgiBot Genie-1, simulated Galaxea R1 Pro, and Unitree G1. The release also states that out-of-distribution language following remains challenging, even after the reported improvements.^[20]

The 2026 Helix 02 and GR00T N1.7 releases emphasize different scaling paths. Helix 02 adds a fast whole-body controller and more sensor modalities, including palm cameras and tactile sensors, within one proprietary humanoid stack.^[21] GR00T N1.7 uses human egocentric video to pretrain an open reasoning VLA intended for fine-tuning across humanoid embodiments.^[22] Neither announcement establishes general-purpose humanoid autonomy outside its disclosed tasks.

Evaluation

What is measured

The most common metric is task success rate, the fraction of episodes that satisfy a predefined goal. Studies may also report partial progress, completion time, intervention rate, path length, action smoothness, collision count, force limits, inference latency, and memory use. A success metric is meaningful only with a task definition, initial-state distribution, number of trials, and criterion for judging completion.

Generalization can refer to different changes:

• Object generalization: unfamiliar object instances or categories.
• Scene generalization: new backgrounds, layouts, lighting, or camera viewpoints.
• Instruction generalization: new phrasing or semantically related commands.
• Skill generalization: a requested behavior absent from the robot demonstrations.
• Embodiment generalization: transfer to a robot with different morphology, sensors, or action space.

These are not interchangeable. A model that handles a new color or prompt wording has not necessarily learned a new motor skill.

LIBERO

LIBERO was introduced as a lifelong robot-learning benchmark with 130 language-conditioned manipulation tasks and human teleoperation demonstrations. The original four suites are LIBERO-Spatial with 10 tasks, LIBERO-Object with 10, LIBERO-Goal with 10, and LIBERO-100 with 100. LIBERO-100 is divided into LIBERO-90 for pretraining and a 10-task downstream set that later VLA literature commonly calls LIBERO-Long or LIBERO-10.^[25]

LIBERO is reproducible and inexpensive relative to physical evaluation, but scores depend on the dataset conversion, observation setup, action normalization, rollout horizon, number of checkpoints, and aggregation rule. A result from one implementation should not be placed beside another without checking those details.

SIMPLER and real-to-sim evaluation

SIMPLER evaluates policies trained on real robot data inside purpose-built simulations of Google Robot and WidowX BridgeData V2 setups. Its authors compared real and simulated performance for several open policies over approximately 1,500 evaluation episodes. They reported strong correlations when control and visual differences were reduced, particularly under the Visual Matching protocol.^[26]

SIMPLER is intended as a scalable proxy and diagnostic tool, not a replacement for physical trials. A simulator may misrepresent contact, friction, camera artifacts, or object dynamics. Correlation between rankings in a particular set of tasks does not imply that simulated success equals real success numerically.

Real-robot evidence

Real-robot tests capture hardware and environmental effects that simulation omits, but are difficult to standardize. Studies use different robots, workcells, teleoperation data, reset conditions, prompts, and human judgment. Some report hundreds or thousands of trials; company announcements may show selected demonstrations without a complete denominator.

Cross-paper rankings are therefore usually invalid. The 16.5-point OpenVLA comparison is meaningful because the authors evaluated OpenVLA and RT-2-X under one protocol.^[8] The OpenVLA-OFT LIBERO and ALOHA improvements are meaningful against the baselines implemented in that paper.^[12] Neither number establishes superiority to later systems tested elsewhere.

The lack of standard protocols is documented in recent reviews of VLA deployment and data infrastructure. They identify inconsistent task definitions, success criteria, data splits, hardware, and scene construction as obstacles to reproducible comparison.^[2][27]

Capabilities and evidence limits

VLA research has demonstrated several capabilities under controlled conditions:

• Language can select among many manipulation behaviors learned from a shared demonstration corpus.
• VLM pretraining can improve visual and semantic generalization relative to robot-only baselines in some evaluations.^[1][8]
• Continuous action experts can generate smooth, high-frequency chunks for bimanual and dexterous control.^[9][13]
• Multi-embodiment training can transfer some behaviors across related and distinct robots in evaluated settings.^[6][19]
• Hierarchical policies can combine slow semantic processing with faster motor control.^[14][21]
• Smaller models can run locally and reduce dependence on a network connection.^[16][17]

Each capability has a boundary. A pretrained VLM may recognize an object without knowing how the target robot can grasp it. An action expert may generate smooth motion that is directed at the wrong object. A multi-robot model can still require embodiment-specific fine-tuning and normalization. A long selected demonstration does not reveal the rate of failed attempts.

The term generalist normally means broader than a task-specific baseline within a study. It does not mean a policy has human-level physical competence or can perform every natural-language request. The published evidence reviewed here did not establish reliable control across arbitrary robots, homes, factories, and instructions.

Limitations

Data scarcity and coverage

Published robot demonstration corpora are far smaller than web-scale text and image corpora. Robot data is costly to collect, and a trajectory is tied to a body, controller, camera setup, and environment. Large mixed datasets can still contain narrow skill coverage, inconsistent labels, and few failures. Human video and simulation improve scale but add embodiment and reality gaps.^[27]

Distribution shift

Changes in lighting, clutter, viewpoint, object geometry, material, and dynamics can change policy behavior. Closed-loop execution compounds error: an early mistake moves the robot into a state absent from demonstrations, making later predictions less reliable. Recovery requires examples of imperfect approaches, failed grasps, and corrective actions, which successful demonstrations may omit.

Long-horizon behavior

Action chunks cover seconds rather than an entire household or industrial task. Longer behavior needs subtask selection, memory, progress estimation, error detection, and recovery. π0.5 and Gemini Robotics 1.5 incorporate semantic subtasks or reasoning to address this problem, but their results remain task- and platform-specific.^[11][19]

Embodiment transfer

Two robots may differ in degrees of freedom, reach, gripper geometry, proprioception, and control frequency. A normalized end-effector action space does not capture every constraint. Motion Transfer, latent-action labeling, and embodiment adapters are active approaches, but there is no solved representation that guarantees zero-shot transfer.^[13][19][27]

Latency and deployment cost

Large VLMs may be too slow or memory-intensive for embedded hardware. Action chunking, asynchronous inference, quantization, smaller backbones, and hierarchical control reduce the burden. They introduce tradeoffs between responsiveness, semantic capacity, smoothness, and compute. Reported frequency should identify whether it refers to model inference, action execution, or a low-level controller, because these can differ by two orders of magnitude.^[1][14][21]

Reproducibility and access

Open weights do not necessarily include the full pretraining data, robot hardware, calibration, or evaluation cell. Closed systems may disclose neither parameter count nor training corpus. Licensing also differs among code, model weights, and datasets. Describing a model simply as open source can obscure those distinctions.

Safety

A VLA is not a complete robot safety system. Its predictions pass through actuators that can exert force, collide, exceed a workspace, or interact with people and fragile objects. Safety depends on the complete stack, including hardware limits, low-level controllers, perception, state estimation, emergency stops, operating procedures, and task-specific validation.

The Gemini Robotics report separates traditional physical action safety from semantic action safety. It states that the VLA should interface with safety-critical lower-level controllers for constraints such as workspace bounds, collision avoidance, and force control. The authors also evaluate an embodied-reasoning model on their ASIMOV semantic-safety datasets and describe constitutional post-training. Those experiments are an initial study of one system, not a guarantee that the action policy will remain safe under all inputs.^[15]

Safety evaluation must distinguish safe failure, unsafe failure, safe success, and unsafe success. A robot can complete a task while using excessive force or entering a hazardous region. ForesightSafety-VLA, a 2026 preprint benchmark, formalizes 13 safety categories spanning physical interaction, language-side hazards, and perception-side hazards. Its simulation experiments use cumulative safety cost and risk-exposure time in addition to endpoint success. The benchmark's authors report nontrivial safety cost even for the strongest baselines they tested, particularly under structural and visual changes. These findings are limited to the benchmark's 66 base scenarios and five simulated embodiments.^[28]

Practical deployments should retain independent constraints beneath the learned policy, evaluate the complete system on the intended hardware, and monitor near-misses rather than only completed tasks. Safety claims based on selected videos or task success alone are insufficient.

Openness and deployment

Public releases for OpenVLA, Octo, the π0 family, the GR00T family, and SmolVLA provide varying combinations of weights, inference code, fine-tuning code, and datasets. LeRobot supplies a PyTorch-based ecosystem and dataset format used by SmolVLA and other policies. Availability supports reproducibility, but code, model weights, and data can have different licenses. Users must examine the license and release notes for each artifact rather than treating open source, open weight, and commercially licensed as interchangeable.

Closed systems include RT-2, Gemini Robotics, and Figure's Helix family. Their papers or technical posts can still disclose architecture and evaluations, but independent replication is limited without weights, training data, and comparable hardware.

Local deployment can reduce network latency and keep sensor data on the robot. SmolVLA and Gemini Robotics On-Device represent open and closed approaches to this goal.^[16][17] Local inference does not by itself establish real-time behavior or safety; the result depends on accelerator speed, camera pipeline, chunk length, controller rate, and the robot's dynamics.

As of 28 July 2026, VLA development remained a rapidly changing research area. The strongest evidence comes from direct comparisons under shared protocols, repeated physical trials, released code and checkpoints, and clearly documented failure cases. Company demonstrations and self-reported benchmark records can identify technical directions, but require independent reproduction before being treated as general capability.

See also

• Embodied AI
• Foundation Models
• Google DeepMind
• Isaac GR00T
• Physical Intelligence
• Robot learning

References

1. ^Zitkovich, B. et al. (2023). "RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control." *Proceedings of The 7th Conference on Robot Learning*, PMLR 229:2165-2183. proceedings.mlr.press/...zitkovich23a
2. ^Kawaharazuka, K., Oh, J., Yamada, J., Posner, I., and Zhu, Y. (2025). "Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications." arXiv:2510.07077. arxiv.org/...2510.07077
3. ^Reed, S. et al. (2022). "A Generalist Agent." *Transactions on Machine Learning Research*. openreview.net/forum
4. ^Brohan, A. et al. (2023). "RT-1: Robotics Transformer for Real-World Control at Scale." *Robotics: Science and Systems XIX*, paper 025. roboticsproceedings.org/...p025
5. ^Driess, D. et al. (2023). "PaLM-E: An Embodied Multimodal Language Model." *Proceedings of the 40th International Conference on Machine Learning*, PMLR 202:8469-8488. proceedings.mlr.press/...driess23a
6. ^Open X-Embodiment Collaboration et al. (2024). "Open X-Embodiment: Robotic Learning Datasets and RT-X Models." *2024 IEEE International Conference on Robotics and Automation*. doi.org/...ICRA57147.2024.10611477
7. ^Octo Model Team et al. (2024). "Octo: An Open-Source Generalist Robot Policy." *Robotics: Science and Systems XX*, paper 090. roboticsproceedings.org/...p090
8. ^Kim, M. J. et al. (2025). "OpenVLA: An Open-Source Vision-Language-Action Model." *Proceedings of The 8th Conference on Robot Learning*, PMLR 270. proceedings.mlr.press/...kim25c
9. ^Black, K. et al. (2025). "π0: A Vision-Language-Action Flow Model for General Robot Control." *Robotics: Science and Systems XXI*, paper 010. roboticsproceedings.org/...p010
10. ^Pertsch, K. et al. (2025). "FAST: Efficient Action Tokenization for Vision-Language-Action Models." *Robotics: Science and Systems XXI*, paper 012. roboticsproceedings.org/...p012
11. ^Black, K. et al. (2025). "π0.5: a Vision-Language-Action Model with Open-World Generalization." *Proceedings of The 9th Conference on Robot Learning*, PMLR 305:17-40. proceedings.mlr.press/...black25a
12. ^Kim, M. J. et al. (2025). "Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success." *Robotics: Science and Systems XXI*, paper 017. roboticsproceedings.org/...p017
13. ^Bjorck, J. et al. (2025). "GR00T N1: An Open Foundation Model for Generalist Humanoid Robots." arXiv:2503.14734. arxiv.org/...2503.14734
14. ^Figure AI. (2025). "Helix: A Vision-Language-Action Model for Generalist Humanoid Control." figure.ai/...helix
15. ^Gemini Robotics Team. (2025). "Gemini Robotics: Bringing AI into the Physical World." arXiv:2503.20020. arxiv.org/...2503.20020
16. ^Shukor, M. et al. (2025). "SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics." arXiv:2506.01844. arxiv.org/...2506.01844
17. ^Google DeepMind. (2025). "Gemini Robotics On-Device brings AI to local robotic devices." deepmind.google/...ngs-ai-to-local-robotic-devices
18. ^NVIDIA Research. (2025). "GR00T N1.5: An Improved Open Foundation Model for Generalist Humanoid Robots." research.nvidia.com/...gr00t-n1_5
19. ^Gemini Robotics Team. (2025). "Gemini Robotics 1.5: Pushing the Frontier of Generalist Robots with Advanced Embodied Reasoning, Thinking, and Motion Transfer." arXiv:2510.03342. arxiv.org/...2510.03342
20. ^NVIDIA Research. (2025). "GR00T N1.6: An Improved Open Foundation Model for Generalist Humanoid Robots." research.nvidia.com/...gr00t-n1_6
21. ^Figure AI. (2026). "Introducing Helix 02: Full-Body Autonomy." figure.ai/...helix-02
22. ^NVIDIA. (2026). "NVIDIA Isaac GR00T N1.7: Open Reasoning VLA Model for Humanoid Robots." huggingface.co/...gr00t-n1-7
23. ^Walke, H. et al. (2023). "BridgeData V2: A Dataset for Robot Learning at Scale." *Proceedings of The 7th Conference on Robot Learning*, PMLR 229. Published proceedings PDF. proceedings.mlr.press/...walke23a.pdf
24. ^DROID project. (updated 2025). "DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset." Current dataset and project record. droid-dataset.github.io
25. ^Liu, B. et al. (2023). "LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning." *NeurIPS 2023 Datasets and Benchmarks Track*. proceedings.neurips.cc/...-Datasets_and_Benchmarks
26. ^Li, X. et al. (2025). "Evaluating Real-World Robot Manipulation Policies in Simulation." *Proceedings of The 8th Conference on Robot Learning*, PMLR 270:3705-3728. proceedings.mlr.press/...li25c
27. ^Wang, Z. et al. (2026). "Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines." arXiv:2604.23001. arxiv.org/...2604.23001
28. ^Lyu, M. et al. (2026). "ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models." arXiv:2606.27079. arxiv.org/...2606.27079
29. ^Chi, C. et al. (2025). "Diffusion Policy: Visuomotor Policy Learning via Action Diffusion." *The International Journal of Robotics Research*, 44(10-11). doi.org/...02783649241273668
30. ^Khazatsky, A. et al. (2024). "DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset." *Robotics: Science and Systems XX*, paper 120. roboticsproceedings.org/...p120