The Best Robot Manipulation Datasets for Commercial Use: A License-First Guide

The short answer: read the license before the leaderboard

When a manipulation dataset trends, the discussion is almost always about scale and benchmark numbers — trajectory counts, embodiment diversity, success rates on a fixed suite. For a research paper that ordering is fine. For a product you intend to ship, it is backwards. The first question is not "how good is this data," it is "am I allowed to train a commercial policy on it and keep the weights." A dataset can sit at the top of every generalization chart and still be off-limits for a company robot because its license reserves commercial rights, or because it is really a bundle of datasets with a dozen different licenses stapled together.

Sort the public manipulation datasets into four license classes and most of the confusion disappears. Permissive open-source licenses (MIT, Apache-2.0, BSD, CC BY) let you train, ship, and keep the model as long as you preserve attribution. Non-commercial licenses (CC BY-NC) let you experiment but block a commercial product without a separate agreement. Gated or custom-agreement datasets require you to sign terms before you ever download, and those terms may restrict downstream use. And aggregated datasets carry no single license at all — the license lives on each contributing subset. Get the class right first; the benchmark comparison only matters among datasets you are actually cleared to use.

Commercial-safe by default: the permissively licensed sets

A handful of well-known manipulation datasets carry genuinely permissive licenses, which is what makes them safe starting points for a commercial policy. BridgeData V2 from Berkeley's RAIL lab ships under the MIT License, and the LIBERO lifelong-learning benchmark is also MIT. MIT is about as unambiguous as commercial licensing gets: you can use, modify, and distribute the material in a product, including closed-source, as long as you keep the copyright and permission notice. DROID sits in the same practical bucket — its authors open-source the full dataset of 76,000 demonstration trajectories (350 hours) captured across 564 scenes and 84 tasks by 50 collectors on three continents, which is one of the largest in-the-wild manipulation corpora you can pull without a bespoke agreement.

The honest caveat is that "commercially licensed" and "useful for your robot" are different tests. DROID is Franka-arm data in largely research and household scenes; BridgeData V2 is a WidowX tabletop; LIBERO is a simulation-anchored benchmark. If your product runs a different arm, gripper, camera placement, or environment, a permissive license clears the legal path but not the embodiment gap. These sets are excellent for pretraining and for validating that your training stack works end to end. They are rarely sufficient on their own for a policy that has to hold up on your specific hardware in your specific deployment.

Attractive but restricted: the non-commercial and gated sets

The datasets that trip up commercial teams are usually the egocentric and human-activity corpora that look perfect for imitation learning and human-to-robot transfer. EPIC-KITCHENS is the canonical example. It is a large, richly annotated first-person cooking dataset that shows up constantly in manipulation and hand-object-interaction work — and it is released under CC BY-NC 4.0, with a separate commercial license required if you want to use it in a product. That -NC suffix is not cosmetic. The CC BY-NC 4.0 license explicitly forbids using the material for commercial purposes, and "we only used it to pretrain a model we then sold" is exactly the use it is written to prevent.

Ego4D is the other pattern: a custom gated agreement. You cannot simply download it. Obtaining Ego4D or its annotations requires reviewing and executing a license agreement first, and credentials arrive only after that agreement is approved. Gated does not automatically mean "no commercial use," but it does mean the terms govern, and a procurement or legal reviewer has to actually read them before your team builds on the data. The failure mode is predictable: an engineer pretrains on an -NC or gated set during research, the result works, it quietly graduates into the production model, and nobody re-checks the license until a partner's due-diligence questionnaire asks for the provenance of every training corpus.

The aggregation trap: Open X-Embodiment is not one license

Open X-Embodiment is the most widely cited manipulation resource of the last two years, and it is also the one most often mishandled from a rights standpoint. It is assembled from 22 different robots through a collaboration between 21 institutions, demonstrating 527 skills. That structure is the whole point technically — cross-embodiment scale is what makes RT-X policies transfer — but it is also why there is no single "Open X-Embodiment license" to accept. Each contributing dataset arrives with its own terms set by its own lab, and those terms are not uniform. Some components are CC BY, some are more restrictive, and the collection as a whole is a directory of licenses, not one grant.

For a commercial team this changes the workflow entirely. You do not clear Open X-Embodiment; you clear the specific subsets you actually train on, one at a time, and you keep the record of which is which. That is manageable when you use two or three components and skip the rest, and it is a genuine liability when an engineer trains on "OXE" as a monolith and cannot later reconstruct which underlying datasets — and which licenses — ended up in the model. The same caution applies to any bundle marketed as a single download: the license that matters is the one on each source, and "it was in the aggregation" is not a defense.

A license file clears the software, not the people in the footage

Even a clean MIT or CC BY license only settles one layer. It grants you rights to the dataset as a work — the code, the packaging, the annotations. It does not, by itself, establish that the humans recorded in the data consented to commercial use of their likeness, that private homes and workplaces were captured with permission, or that no third-party brand or IP in frame creates a separate problem. For manipulation data this matters more than it sounds, because so much of it is filmed in real kitchens, warehouses, and homes with real people and real products in view.

This is the layer that partner due diligence and enterprise legal review actually probe, and it is where a dataset's license file goes quiet. The CC family draws the commercial line at the license grant, but consent for the people and places in the footage is a distinct chain of documentation the license does not speak to. When you source data rather than scrape it, this becomes a specification requirement: contributor consent artifacts, location releases where applicable, and per-trajectory provenance you can hand an auditor. Truelabel treats that as part of rights-cleared delivery — consent records and per-trajectory metadata travel with the trajectories — precisely because a permissive license on the container is not the same as documented rights on the contents.

When public datasets run out: sourcing licensed data on spec

Public manipulation datasets are a warm start, not a finish line. They are fixed in embodiment, environment, and task distribution, and no amount of favorable licensing changes the fact that a benchmark corpus was not collected for your robot. Evaluation makes the ceiling visible: frameworks like ManipArena — 20 tasks, 10,812 expert trajectories, 13.5M frames, and roughly 188 robot hours across tabletop and mobile manipulation — exist because policies that look strong on one dataset often fail under matched physical conditions on different hardware. The moment your policy needs your gripper, your camera rig, your objects, and your edge cases, you are past what any public set can supply.

That is where the sourcing path takes over from the download path. Instead of accepting whatever license and distribution a public dataset happens to carry, you write a spec — embodiment, viewpoint, modalities, action schema, edge cases, reject conditions — and commission data collected against it with commercial rights defined up front. Truelabel is built for that side: a physical AI data marketplace where robotics teams post a spec and matched partners return sample packets, drawing on 100+ vetted capture partners and a research catalog of 750+ profiled public and commercial physical-AI datasets. Deliveries arrive rights-cleared in RLDS, LeRobot, MCAP, or your own schema, with QA evidence and consent artifacts included, so the license question is answered by contract before the data reaches training rather than discovered during a customer's audit.

Related pages

Use these to move from category-level context into specific task, dataset, format, and comparison detail.

FAQ