OpenVLA: Open-Source Vision-Language-Action Model for Robot Manipulation

Quick facts

Topic

OpenVLA

Audience

Robotics engineers, ML ops, procurement leads

Deliverable

Buyer-facing model reference + sourcing guidance

What OpenVLA is and why the open weights matter

OpenVLA is the first open-weight vision-language-action model that holds its own against closed frontier robot policies^[1]. It takes one RGB frame and a language instruction and predicts a robot action, and its 7 billion parameters are released under Apache 2.0, so the weights can go into a commercial product without a licensing negotiation over the model itself.

The headline result is efficiency, not just openness. Across 29 real-robot tasks OpenVLA beats RT-2-X, a 55-billion-parameter closed model, by 16.5 percentage points in absolute success rate while carrying 7x fewer parameters, and it outperforms a from-scratch Diffusion Policy by 20.4% in multi-task settings^[1]. For a procurement team, that combination — open weights, small enough to fine-tune in-house, competitive with models an order of magnitude larger — is what makes OpenVLA the default baseline before committing budget to a proprietary policy.

The open weights do not launder the data licenses underneath them. OpenVLA's pretraining set is drawn from the Open X-Embodiment collection, which aggregates dozens of constituent datasets under heterogeneous terms, some permissive and some non-commercial. Apache 2.0 governs the checkpoint; it says nothing about whether fine-tuning on a non-commercial constituent propagates that restriction to your derivative. That question is a legal review, not a benchmark, and it is the one buyers most often skip.

Architecture: fused DINOv2 + SigLIP vision, discretized action tokens

OpenVLA is built on the Prismatic vision-language recipe. Its visual front end fuses features from two pretrained encoders, DINOv2 and SigLIP, concatenating them channel-wise so the model gets DINOv2's spatial, geometric features alongside SigLIP's language-aligned semantics^[1]. Those visual tokens are prepended to the tokenized instruction and fed to a Llama 2 7B backbone, which predicts the action autoregressively, exactly as a language model predicts the next word.

The action representation is the detail worth getting right, because it is the one most often misreported. OpenVLA does not use a diffusion head and does not regress continuous values in the base model. It discretizes each of the seven end-effector dimensions — three for translation, three for rotation, one for the gripper — into 256 bins, then maps those bins onto the 256 least-frequently-used tokens in the Llama vocabulary^[1]. Predicting an action is therefore literally next-token prediction over that repurposed vocabulary, the same tokenization lineage as RT-1 and RT-2.

That design buys simplicity and pays for it in speed. Emitting seven tokens one at a time, autoregressively, caps the base model at only a few actions per second on a single GPU — fine for quasi-static pick-and-place, too slow for pouring, insertion, or anything where control lag compounds. Closing that gap is the entire motivation for OpenVLA-OFT below.

Training data: 970K Open X-Embodiment demonstrations in RLDS

OpenVLA is pretrained on a curated mixture from the Open X-Embodiment collection, roughly 970,000 real-world robot demonstrations aggregated from dozens of research labs^[1][2]. The corpus ships in RLDS, a TensorFlow-based schema that stores each episode as a sequence of (observation, action, reward, discount) steps^[3], which is why RLDS fluency is a hard prerequisite for anyone fine-tuning OpenVLA rather than a nice-to-have.

Each trajectory pairs an RGB observation, a natural-language task string, and a normalized action vector. The normalization statistics — per-dimension mean and range used to map raw actions into the model's expected scale — are load-bearing. If the statistics computed on your downstream data drift from what the model was trained against, predicted deltas come out mis-scaled and the policy overshoots or stalls, a failure that looks like a model problem but is really a data-pipeline problem.

For teams capturing new data, the format imposes concrete shape requirements: uint8 RGB arrays, float32 action vectors at a fixed dimensionality, and instruction strings attached per episode. Varying the phrasing of those instructions across demonstrations of the same task is worth the effort — BridgeData V2 documents that language and scene diversity in the training set is what makes the resulting policy robust to how a real operator actually phrases a command^[4].

Fine-tuning: LoRA on a single GPU, and why quality beats volume

OpenVLA can be adapted either by full fine-tuning or by LoRA, low-rank adaptation that updates a small set of injected weights and freezes the rest^[1]. LoRA is what brings fine-tuning down to a single consumer-class GPU, which is the practical reason a small team can specialize OpenVLA to its own robot without a training cluster. The model adapts to a new task or embodiment on the order of tens to a few hundred demonstrations, scaling with task complexity and how far the target visual domain sits from the pretraining distribution.

Beyond a modest floor, demonstration quality moves success more than demonstration count. The DROID program, built explicitly to study this, shows that consistency — stable camera placement, controlled lighting, competent teleoperation — separates a dataset that trains a working policy from one that merely looks large^[5]. The procurement consequence is direct: budget for operator training, calibration, and a QA gate, not just for a raw trajectory target, because a few hundred clean, consistent demonstrations routinely beat a larger pile collected under drifting conditions.

OpenVLA-OFT: parallel decoding for high-frequency control

The base model's token-by-token decoding is the bottleneck OpenVLA-OFT sets out to remove. OFT (Optimized Fine-Tuning) is a separate 2025 recipe that changes how the policy emits actions: it combines parallel decoding, action chunking, a continuous action representation, and a simple L1-regression objective, replacing the slow autoregressive discrete-token loop^[6]. The reported result is a 26x increase in action-generation throughput over base OpenVLA, and on the LIBERO benchmark it lifts the four-suite average success rate from 76.5% to 97.1%, enough to run dexterous, high-frequency control on a bimanual ALOHA setup^[6].

That speed shifts the data requirement. A policy running fast, contact-rich control needs demonstrations captured and time-aligned at correspondingly higher rates, because chunked action prediction only helps if the training trajectories actually carry the temporal resolution the controller will run at. Sourcing for an OFT-style deployment therefore means teleoperation with tight cross-sensor synchronization, not a folder of loosely-timed clips — the same synchronization discipline any high-Hz manipulation program lives or dies on.

OpenVLA vs RT-2-X and Octo

The three open-and-adjacent generalist policies split cleanly on openness, size, and how they produce actions. RT-2-X is the closed, 55-billion-parameter web-scale option: strong semantic generalization from internet vision-language pretraining, but unavailable to deploy or fine-tune. Octo is the small, fast open option — an Octo-Base around 93M parameters trained on roughly 800,000 Open X-Embodiment trajectories, using a diffusion action head rather than discrete tokens, and cheap enough to fine-tune quickly^[7]. OpenVLA sits in between: open like Octo, but 7B parameters and discretized-token actions, trading Octo's speed for higher absolute success.

The decision is budget-shaped. If GPU budget is tight and iteration speed matters more than the last few points of success, Octo is the accessible entry point. If absolute performance and cross-embodiment transfer justify the compute, OpenVLA is the stronger baseline, and OpenVLA-OFT recovers much of the speed Octo would otherwise win on. RT-2-X is the reference ceiling you compare against but cannot ship.

Sourcing and integrating OpenVLA training data

A fine-tuning pipeline for OpenVLA almost always stores episodes in RLDS or converts to it, and LeRobot provides the loaders that turn those episodes into batched training data across PyTorch, so the format decision is mostly about who has already tooled which schema^[8]. The hard part of sourcing is never raw volume; it is synchronization and consistency. Frames, proprioceptive state, and gripper signals have to share a clock tight enough that each action lines up with the observation that produced it, and camera placement has to stay stable enough that the model's learned visual features still apply — a few centimeters or a few milliseconds of drift quietly degrades the policy.

This is the specification Truelabel's physical AI data marketplace is built to fill: multi-view teleoperation and egocentric capture from around 10,000 consented collectors across 100 countries, delivered in RLDS or LeRobot format to S3, GCS, or Azure with camera calibration, temporal alignment, and per-trajectory provenance^[9]. Sample packets ship before scale so the calibration and QA gate run on a real batch first, and per-episode provenance records let a buyer audit consent and trace a bad batch back to its source — the documentation a commercial deployment needs and a research-licensed public set cannot provide. For deeper context on the model family and the paired signal it consumes, see the vision-language-action model definition and the VLA training data hub.

Related pages

Use these to move from category-level context into specific task, dataset, format, and comparison detail.

External references and source context

1. OpenVLA: An Open-Source Vision-Language-Action Model

   OpenVLA (arXiv 2406.09246): 7B parameters; fused DINOv2 + SigLIP vision encoder on a Llama 2 7B backbone; trained on 970K real-world robot demonstrations from Open X-Embodiment; discretized action tokens (256 bins per dimension mapped onto least-used Llama vocabulary tokens); outperforms RT-2-X (55B) by 16.5 percentage points absolute across 29 tasks with 7x fewer parameters; beats Diffusion Policy by 20.4% multi-task; LoRA fine-tuning on a single consumer GPU; Apache 2.0 weights.

   arXiv ↩
2. Open X-Embodiment: Robotic Learning Datasets and RT-X Models

   Open X-Embodiment: the aggregated multi-dataset robot-learning collection from which OpenVLA's ~970K-demonstration pretraining mixture is drawn, distributed in RLDS with heterogeneous constituent licenses.

   arXiv ↩
3. RLDS: an Ecosystem to Generate, Share and Use Datasets in Reinforcement Learning

   RLDS format specification: episodes stored as sequences of (observation, action, reward, discount) steps, the schema OpenVLA training data uses.

   arXiv ↩
4. BridgeData V2: A Dataset for Robot Learning at Scale

   BridgeData V2: language and scene diversity in the training set improves policy robustness to how commands are phrased and to scene variation.

   arXiv ↩
5. DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset

   DROID: demonstration consistency (stable camera placement, controlled lighting, competent teleoperation) matters more than raw demonstration count for policy quality.

   arXiv ↩
6. Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success

   OpenVLA-OFT (arXiv 2502.19645): parallel decoding, action chunking, continuous action representation, and L1-regression objective yield a 26x increase in action-generation throughput over base OpenVLA and raise the LIBERO four-suite average success rate from 76.5% to 97.1%, enabling high-frequency bimanual ALOHA control.

   arXiv ↩
7. Octo: An Open-Source Generalist Robot Policy

   Octo: open-source generalist robot policy; Octo-Base around 93M parameters trained on roughly 800K Open X-Embodiment trajectories using a diffusion action head, positioned as the smaller, faster open alternative to OpenVLA.

   arXiv ↩
8. LeRobot documentation

   LeRobot framework: dataset loaders and tooling for turning RLDS/LeRobot episodes into batched PyTorch training data.

   Hugging Face ↩
9. truelabel physical AI data marketplace bounty intake

   Truelabel physical AI data marketplace: multi-view teleoperation and egocentric capture from around 10,000 consented collectors across 100 countries, delivered in RLDS or LeRobot to S3/GCS/Azure with calibration, temporal alignment, provenance, and sample packets before scale.

   truelabel.ai ↩

FAQ