Citation and evidence

OpenVLA

17 min full readUpdated 19 references

This article's verification

Report a problem with this article

Edit article
More

Use this article

Raw MarkdownExplore connections

Improve this page

Revision historyDiscuss this article

Topics

AI ModelsOpen Source AIRobotics

Cite this article

OpenVLA is a 7-billion-parameter open-source vision-language-action model (VLA) for robotic manipulation, released in June 2024 by a collaboration of researchers from Stanford University, UC Berkeley, the Toyota Research Institute, Google DeepMind, Physical Intelligence, and MIT.[1] It was the first generalist robot foundation model of its size to ship with public weights, training code, and a permissive license, and it outperformed the closed 55-billion-parameter RT-2-X by 16.5 percentage points in absolute task success across 29 tasks while using 7 times fewer parameters.[1] It set a reference point for what an open VLA could look like in an area that had been dominated by closed industrial models such as RT-2.[1]

Related: Llama 2π0Octo (Robot Policy)Open X-Embodiment

The model is built on a Prismatic VLM backbone that fuses DINOv2 and SigLIP visual encoders with a Llama 2 7B language model.[1][5] It was trained on roughly 970,000 robot manipulation trajectories drawn from the Open X-Embodiment dataset, covering more than 70 individual robot datasets and 22 embodiments.[1][7] The authors summarize the result this way: OpenVLA "outperform[s] closed models such as RT-2-X (55B) by 16.5% in absolute task success rate across 29 tasks and multiple robot embodiments, with 7x fewer parameters."[1] The paper was accepted at the Conference on Robot Learning (CoRL) 2024.[1]

OpenVLA matters less because of any single benchmark number and more because of what it unlocked. Within months of release, dozens of follow-ups began fine-tuning it on new arms, distilling it into smaller policies, retrofitting it with continuous action heads, and using it as the baseline that any new VLA had to beat. The 2025 sequel OpenVLA-OFT pushed the same checkpoint to what was, in February 2025, state of the art on the LIBERO simulation benchmark and made bimanual high-frequency control practical on ALOHA-style hardware.[10]

What is a vision-language-action model?

Vision-language-action models are a class of foundation models that take an image (or several) plus a natural-language instruction and output low-level robot actions, usually end-effector deltas or joint targets. The lineage runs through Google's RT-1 (2022), which was a small transformer policy trained on robot demonstrations, and RT-2 (2023), which fine-tuned a closed vision-language model (PaLI-X, up to 55B parameters) to emit action tokens directly.[6] RT-2 showed that internet-scale visual and linguistic priors transfer to manipulation in ways pure robot data cannot match, but it was never released.[6] RT-X / RT-2-X, announced alongside the Open X-Embodiment collaboration in late 2023, retrained RT-2 on the cross-embodiment dataset and reported strong generalization across robots, again without releasing weights.[7]

The situation by spring 2024 was awkward. The strongest generalist policies were closed and ran on Google's internal infrastructure. Open alternatives existed (notably Octo from UC Berkeley, a 27M to 93M parameter transformer trained on the same Open X-Embodiment data), but they did not use a pretrained language model as the backbone and lagged on language-conditioned generalization.[8] OpenVLA was designed to close that gap: take the recipe RT-2 had validated, swap in an open VLM, train on the same public data, and release everything.[1]

"OpenVLA: LeRobot Research Presentation #5 by Moo Jin Kim", video from Hugging Face, published August 13, 2024. Watch on YouTube.

How is OpenVLA built?

OpenVLA is, mechanically, a fine-tuned Prismatic VLM.[1] The Prismatic family was introduced earlier in 2024 by Karamcheti and collaborators as an open recipe for vision-language model training; OpenVLA adopts the prism-dinosiglip-224px configuration.[5] Three components matter.

OpenVLA diagram showing DINOv2 and SigLIP image encoders, an MLP projector, a language tokenizer, Llama 2 7B and an action de-tokenizer

Architecture of the original OpenVLA model: visual features and a language instruction feed a Llama 2 7B backbone, whose output tokens are decoded into robot actions.[17] Figure: Moo Jin Kim and coauthors. CC BY 4.0.

Vision encoder

Each RGB image is fed through two pretrained vision encoders in parallel. SigLIP (a CLIP-style sigmoid-loss image-text model from Google) provides features that are well aligned with language.[14] DINOv2 (Meta's self-supervised vision transformer) provides features that are stronger for spatial and geometric reasoning.[13] The two feature maps are concatenated channel-wise. This dual-encoder setup is one of the Prismatic recipe's key contributions: it gave better grounding than either encoder alone.[5]

Projector

A two-layer MLP projects the fused visual features into the language model's embedding space, where they are treated as if they were tokens.[1]

Language model

The backbone is Llama 2 7B.[12] It autoregressively predicts a sequence of tokens, conditioned on the visual embeddings followed by the natural-language instruction. The output sequence is interpreted as a robot action.[1]

Action tokenization

This is where OpenVLA diverges from a typical VLM. Robot actions are continuous (7 degrees of freedom for a single arm: 3 translation, 3 rotation, 1 gripper), but Llama 2 emits discrete tokens. The authors discretize each action dimension into 256 uniform-width bins spanning the 1st to 99th percentile of that dimension in the training data; using percentiles rather than the min and max (as RT-2 did) keeps outlier actions from stretching the bins.[1]

Rather than expand the vocabulary, OpenVLA overwrites the 256 least-used tokens in the Llama 2 tokenizer (numerically, the rarest BPE tokens) and reassigns them to action bins.[1] At inference time the model produces 7 of these tokens per timestep, which a small de-tokenizer maps back into a continuous action vector that the robot can execute.[1] The trick is cheap, simple, and lets the entire VLM remain unchanged below the embedding layer.

The input image resolution is 224x224.[1] The model takes a single third-person camera frame at each step (no proprioception, no history), which keeps things simple but also limits the kinds of tasks the base model can solve.[1]

How was OpenVLA trained?

Data

The pretraining mixture is a curated subset of Open X-Embodiment, the cross-institution dataset that aggregates over 70 robot datasets and roughly 2 million trajectories from 22 embodiments.[7] The OpenVLA team filtered this down to about 970,000 trajectories.[1] The filtering kept only manipulation datasets with at least one third-person camera and single-arm end-effector control, then reused Octo's mixture weights, which down-weight or drop less diverse datasets and up-weight more diverse ones.[1] The action space is unified to a single 7-DoF end-effector Cartesian formulation; per-dataset proprioceptive and gripper conventions are normalized.[1]

Compute

OpenVLA-7B was trained on 64 NVIDIA A100 GPUs for roughly 14 days.[1] Total compute was about 21,500 A100-hours at a global batch size of 2,048.[1] By 2024 foundation model standards, that is small. By robotics standards, it is enormous; most published manipulation policies before OpenVLA used a few thousand GPU-hours at most.

Optimization

The training objective is plain next-token cross-entropy on the action token sequence.[1] There is no separate action loss, no diffusion head, no flow-matching component (those approaches came later, in pi0 and OpenVLA-OFT).[10] The full Llama 2 backbone is updated.[1] The vision encoders are fine-tuned along with the rest of the model: unlike typical VLM training, the authors found that updating the vision encoder was crucial for VLA performance (80.0% vs 46.7% average success in their ablation).[1]

How does OpenVLA compare to RT-2-X?

The original paper reports zero-shot evaluation on 29 tasks across multiple robots, comparing OpenVLA-7B to RT-1-X, Octo, and the closed RT-2-X (55B).[1] Results are aggregated by embodiment.

ModelParametersOpen weightsBridgeData V2 (WidowX)Google robot
RT-1-X35Myes18.5%33.3%
Octo93Myes20.0%26.7%
RT-2-X55Bno50.6%78.3%
OpenVLA-7B7Byes70.6%85.0%

Expanded article table

The headline number is +16.5 percentage points absolute success rate over RT-2-X averaged across the 29 tasks, despite OpenVLA having seven times fewer parameters.[1] On the WidowX BridgeData V2 benchmark in particular, OpenVLA outperforms RT-2-X by a wide margin; on the Google robot evaluation it is roughly comparable.[1] Against Octo and RT-1-X, OpenVLA wins on essentially every task category, with the gap widening on language-conditioned generalization (novel object combinations, novel attribute descriptions).[1]

The paper also reports fine-tuning experiments. Given 10 to 150 demonstrations of a new task on a new robot (Franka Panda is the headline platform), full fine-tuning takes 5 to 15 hours on 8 A100s, and OpenVLA was the only tested method to reach at least 50% success on every task.[1] With LoRA fine-tuning, the same range can be reached in 10 to 15 hours on a single A100 while updating only about 1.4% of parameters, an 8x reduction in training compute over full fine-tuning.[1] The authors observe that LoRA fine-tunes match full fine-tuning quality on the tasks they tested, which put adaptation to a new robot within reach of a single GPU.[1]

How does OpenVLA differ from other VLA models?

ModelYearParametersBackboneAction representationOpen weights
RT-1202235MEfficientNet + transformerDiscrete tokensNo
RT-22023up to 55BPaLI-XDiscrete tokensNo
RT-2-X202355BPaLI-XDiscrete tokensNo
Octo202427M / 93MTransformer + diffusion headContinuous (diffusion)Yes
OpenVLA20247BLlama 2 + DINOv2 + SigLIPDiscrete tokens (256 bins)Yes (MIT code, Llama 2 license on weights)
pi020243.3BPaliGemma + flow-matching action expertContinuous (flow matching)Yes (since Feb 2025)
OpenVLA-OFT20257BOpenVLA + L1 regression headContinuous (action chunks)Yes
GR00T N120252.2BVLM + diffusion transformerContinuous (action chunks)Yes

Expanded article table

A few observations are worth pulling out. Among the open models in this table, OpenVLA is the largest and the only one with a pure autoregressive token decoder. Almost every model that came after it moved to continuous action representations (diffusion or flow matching) for one main reason: faster inference. Token-by-token decoding limits OpenVLA to roughly 3 Hz on an A100 and about 6 Hz on an RTX 4090 or H100 in bfloat16 for a single arm, which is not enough for high-frequency control.[1] The flip side is that the discrete-token formulation lets the VLM be trained essentially as a language model, which is conceptually simple and lets it inherit instruction-following behavior from Llama 2 cleanly.[12]

Against Octo, OpenVLA wins on language understanding and out-of-distribution objects but is two orders of magnitude larger and much slower at inference.[1] Octo's 93M-parameter model fine-tunes in about 5 hours on a single 24 GB NVIDIA A5000 GPU.[8] The choice between them is genuinely a tradeoff, not a strict ordering.

Against pi0, released a few months later by Physical Intelligence, OpenVLA shipped open weights at launch, while pi0's weights were only released in February 2025, through Physical Intelligence's openpi repository, whose code is Apache 2.0 and which also ships the Gemma Terms of Use for the PaliGemma-derived weights.[18][19] pi0 uses flow matching to produce 50 Hz continuous action chunks, which is more practical for dexterous tasks.[9] On the laundry-folding and table-bussing demos that pi0 leaned on, OpenVLA out of the box would not be competitive.[9] With OFT-style fine-tuning, the gap narrows considerably.[10]

Against GR00T N1, released by NVIDIA in March 2025, OpenVLA is older and uses a single-camera, single-arm action interface, while GR00T N1 targets humanoids but is also evaluated across several embodiments.[11] GR00T N1 is humanoid-focused and uses a dual-system architecture (slow planner + fast diffusion action policy), trained on 50,000 H100 GPU-hours, much more compute than OpenVLA used.[11]

Is OpenVLA open source?

Yes. OpenVLA was released with code under the MIT license at https://github.com/openvla/openvla, model weights on Hugging Face at https://huggingface.co/openvla/openvla-7b, training scripts, the data filter and weighting code, and inference servers for several robots.[3][4] The weights are subject to the Llama 2 community license inherited from the backbone, which is permissive for most uses but has Meta's standard restrictions for very large commercial deployments.[12]

The authors emphasize accessibility as a design goal, noting in the paper that OpenVLA models "can be fine-tuned on consumer GPUs via modern low-rank adaptation methods and served efficiently via quantization without a hit to downstream success rate."[1] The practical effect of releasing the entire pipeline (and not just weights) was significant. The authors also showed that 4-bit quantized inference matches bfloat16 while using less than half the GPU memory.[1] The official codebase supports full fine-tuning, partial fine-tuning, and quantized LoRA via Hugging Face's PEFT library out of the box.[3]

How do you fine-tune OpenVLA?

In the paper's experiments, fine-tuning OpenVLA on a new task meant collecting 10 to 150 teleoperated demonstrations on the target robot and running one of three recipes.[1]

Full fine-tuning. Update all parameters, typically with 8 A100 or H100 GPUs for 5 to 15 hours. Best ceiling, most compute.[1]

LoRA fine-tuning. Update low-rank adaptation modules over the attention and MLP layers, typically rank 32. One A100, 10 to 15 hours, comparable success rate on the tasks the authors tested.[1] This is the most common recipe in published follow-ups.

Quantized inference. Serving the fine-tuned model in 4-bit precision cuts inference memory from 16.8 GB to 7.0 GB with BridgeData V2 success matching bfloat16 (71.9% vs 71.3%); 8-bit inference was slower and dropped to 58.1%.[1]

The fine-tuning workflow is one of the main reasons OpenVLA caught on. RT-2 fine-tuning is something only Google can do; OpenVLA fine-tuning is something a small lab can do over a weekend. That accessibility, more than any benchmark number, is what made OpenVLA the default VLA for academic robotics work through 2024 and into 2025.

What are OpenVLA's limitations?

The paper and follow-up work are candid about what OpenVLA does not solve.

Inference throughput. Autoregressive token decoding produces one action per forward pass, with each forward pass spending most of its time on the 7B language model. The paper measures roughly 3 Hz on an A100 and about 6 Hz on an RTX 4090 or H100 for a single arm in bfloat16, well below the 50 Hz that high-frequency setups such as ALOHA run at.[1] This is the limitation that drove most subsequent VLA work.

Single-image, no history. The base model takes one third-person camera frame and no proprioception. Tasks that need wrist cameras, force feedback, or memory of recent states are out of distribution.[1]

Coarse action discretization. 256 bins per dimension is fine for pick-and-place but coarse for fine insertion or pouring tasks. Continuous action heads (added in OpenVLA-OFT, pi0, GR00T) close this gap.[10]

Bimanual and dexterous manipulation. With a single image and slow autoregressive decoding, bimanual tasks running at 50 Hz on an ALOHA cell are essentially impossible at the base recipe. OFT-style fine-tuning was needed to make this work.[10]

Reliability ceiling. Even on tasks where OpenVLA outperforms RT-2-X, success rates typically live in the 70-90% band rather than the 95%+ range that a deployed system would want.[1] Generalist robot policies are not yet at the reliability that, say, language models reached for translation; OpenVLA is no exception.

What came after OpenVLA?

The most directly tied successor is OpenVLA-OFT (Optimized Fine-Tuning), released in February 2025 by Moo Jin Kim, Chelsea Finn, and Percy Liang.[10] OFT is not a new model so much as a fine-tuning recipe applied to the same OpenVLA-7B checkpoint. It does four things at once: replaces the discrete action token head with an L1 regression head producing continuous values, predicts action chunks (multiple future timesteps at once) instead of single actions, decodes those chunks in parallel rather than autoregressively, and fine-tunes with a simple regression objective.[10] The OFT paper reports that this recipe "boosts OpenVLA's average success rate across four LIBERO task suites from 76.5% to 97.1% while increasing action generation throughput by 26x."[10] On real bimanual ALOHA hardware, OpenVLA-OFT outperforms pi0 and RDT-1B fine-tuned with their default recipes, as well as scratch-trained Diffusion Policy and ACT, by up to 15 percentage points on average.[10] With 25-step action chunks, throughput reaches 43x the base model.[10]

Other notable follow-ups include MiniVLA, smaller variants for edge deployment; numerous task-specific fine-tunes released on Hugging Face across the second half of 2024 and 2025; and the use of OpenVLA as a starting point for online reinforcement learning experiments, where a pretrained generalist policy is bootstrapped with on-robot RL. Most VLA papers published after mid-2024 use OpenVLA as a baseline, which is its own kind of citation impact.

Reception and impact

In the months after release, OpenVLA was widely covered in robotics-focused press as the first credibly open competitor to RT-2. VentureBeat called it an "open-source generalist robotics model" and tied it to the broader push to make embodied AI reproducible.[16] The 2024 AI Agent Index includes it as an entry, describing it as an open-source generalist robot policy.[15]

The research-community impact is harder to summarize cleanly because it is still ongoing. Two things are clear. First, almost every open VLA paper since June 2024 either uses OpenVLA's recipe (Prismatic backbone, Open X-Embodiment data filter, action tokenization scheme) or compares against it.[1] Second, the model became the default starting point for graduate students training policies on lab arms; a search of recent CoRL and RSS papers finds dozens of OpenVLA-derived systems. The paper has been cited in many subsequent surveys of VLA models and robot foundation models.

The authors went on to other work in roughly the directions you would expect. Moo Jin Kim led OpenVLA-OFT.[10] Karl Pertsch and others moved closer to Physical Intelligence and the pi0 line. Sergey Levine remains at UC Berkeley, where his group continues releasing open robot policies. Chelsea Finn and Percy Liang co-advised the OFT follow-up.[10] Russ Tedrake's involvement at TRI signaled the institute's continued bet on open VLAs alongside its own internal work.

Authors

The full author list, in order: Moo Jin Kim (Stanford), Karl Pertsch (Stanford / UC Berkeley), Siddharth Karamcheti (Stanford / TRI), Ted Xiao (Google DeepMind), Ashwin Balakrishna (TRI), Suraj Nair (TRI), Rafael Rafailov (Stanford), Ethan Foster (Stanford), Grace Lam (Stanford), Pannag Sanketi (Google DeepMind), Quan Vuong (Physical Intelligence; work done in part at Google DeepMind), Thomas Kollar (TRI), Benjamin Burchfiel (TRI), Russ Tedrake (TRI / MIT), Dorsa Sadigh (Stanford), Sergey Levine (UC Berkeley), Percy Liang (Stanford), Chelsea Finn (Stanford).[1] Lead authorship (Kim, Pertsch, Karamcheti) is shared.[1]

Continue reading

More from See also

Topics:AI ModelsOpen Source AIRobotics

References

  1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12 ^13 ^14 ^15 ^16 ^17 ^18 ^19 ^20 ^21 ^22 ^23 ^24 ^25 ^26 ^27 ^28 ^29 ^30 ^31 ^32 ^33 ^34 ^35 ^36 ^37 ^38 ^39 ^40 ^41 ^42 ^43 ^44 ^45Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., Sanketi, P., Vuong, Q., Kollar, T., Burchfiel, B., Tedrake, R., Sadigh, D., Levine, S., Liang, P., & Finn, C. (2024). *OpenVLA: An Open-Source Vision-Language-Action Model*. arXiv:2406.09246. Conference on Robot Learning (CoRL) 2024. arxiv.org/...2406.09246
  2. OpenVLA project website. openvla.github.io
  3. ^1 ^2OpenVLA GitHub repository. github.com/...openvla
  4. ^OpenVLA-7B model card. Hugging Face. huggingface.co/...openvla-7b
  5. ^1 ^2 ^3Karamcheti, S., et al. (2024). *Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models*. arXiv:2402.07865.
  6. ^1 ^2Brohan, A., et al. (2023). *RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control*. arXiv:2307.15818.
  7. ^1 ^2 ^3Open X-Embodiment Collaboration. (2024). *Open X-Embodiment: Robotic Learning Datasets and RT-X Models*. arXiv:2310.08864.
  8. ^1 ^2Octo Model Team. (2024). *Octo: An Open-Source Generalist Robot Policy*. arXiv:2405.12213.
  9. ^1 ^2Black, K., et al. (2024). *pi0: A Vision-Language-Action Flow Model for General Robot Control*. Physical Intelligence. arXiv:2410.24164.
  10. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12Kim, M. J., Finn, C., & Liang, P. (2025). *Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success* (OpenVLA-OFT). arXiv:2502.19645. arxiv.org/...2502.19645
  11. ^1 ^2NVIDIA Research. (2025). *GR00T N1: An Open Foundation Model for Generalist Humanoid Robots*. arXiv:2503.14734.
  12. ^1 ^2 ^3Touvron, H., et al. (2023). *Llama 2: Open Foundation and Fine-Tuned Chat Models*. Meta AI.
  13. ^Oquab, M., et al. (2023). *DINOv2: Learning Robust Visual Features without Supervision*. Meta FAIR.
  14. ^Zhai, X., et al. (2023). *Sigmoid Loss for Language Image Pre-Training* (SigLIP). Google Research.
  15. ^AI Agent Index. (2024). *OpenVLA*. MIT. aiagentindex.mit.edu/...openvla
  16. ^VentureBeat. (2024). *OpenVLA is an open-source generalist robotics model*. venturebeat.com/...ource-generalist-robotics-model
  17. ^Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. "OpenVLA: An Open-Source Vision-Language-Action Model," Figure 2. arXiv:2406.09246v3, 2024. CC BY 4.0. Unchanged arXiv-hosted PNG rendition of Figure 2; no local image changes.
  18. ^Physical Intelligence. (2025, February 4). "Open Sourcing π0." physicalintelligence.company/...openpi
  19. ^Physical Intelligence. openpi GitHub repository (LICENSE, Apache 2.0, and LICENSE_GEMMA.txt, Gemma Terms of Use). github.com/...openpi

Improve this article

Add missing citations, update stale details, or write a clearer explanation. Anyone can edit: changes go live when you publish, and every edit is checked by an editor.

8 revisions · v8 · 3,464 words · full history

Fact-checks are independent of edits: a reviewer re-verifies the article against its sources and stamps the date. How we verify

Research and drafting on this wiki are AI-assisted, under named human editorial standards. How AI is used here

Reviewer note: Independent fact-check of the full article against its cited sources (110 claims, 19 sources fetched); 10 material + 17 minor fixed

Cite this page: AI Wiki. "OpenVLA." aiwiki.ai, updated 17 Nov 2026, fact-checked 2 Nov 2026. CC BY 4.0. https://aiwiki.ai/wiki/openvla

Edit article