Task data
Teleoperation training data
Teleoperation training data helps physical AI teams collect scoped examples in robot workcells, warehouses, kitchens, and labs. When sourcing it, specify robot state, action traces, and synchronized camera streams, target volume, delivery format, rights, consent, and QA rules for timestamp alignment, state/action completeness, and recoverable failure examples.
Quick facts
- Task
- Teleoperation
- Modality
- robot state, action traces, and synchronized camera streams
- Environment
- robot workcells, warehouses, kitchens, and labs
- Volume
- 20-200 hours of teleoperated episodes
- Format
- MCAP, ROS bag, HDF5, RLDS, or LeRobot
- QA
- timestamp alignment, state/action completeness, and recoverable failure examples
Comparison
| Source | Use | Limitation |
|---|---|---|
| Public dataset | Research baseline | video-only demonstrations cannot train action-producing policies without robot state |
| Internal capture | Maximum control | Slow setup and high fixed cost |
| truelabel sourcing | Spec-matched supplier response | Requires clear acceptance criteria |
What to specify for teleoperation
The sourcing request should define task boundaries, capture setting, actor or robot requirements, accepted modalities, MCAP, ROS bag, HDF5, RLDS, or LeRobot delivery expectations, rights, consent, and what counts as an accepted sample. Registry sources show that task data is only reusable when collection setup and task distribution are explicit [1]. Buyers should also pin delivery expectations to formats and documentation they can validate before scale [2].
Why public data is usually not enough
video-only demonstrations cannot train action-producing policies without robot state. Benchmark and vendor sources show that task labels, rights, and capture context are not interchangeable across deployments [3]. A buyer-specific request lets the team request the exact object set, environment, geography, and QA rubric needed for model training or evaluation.
Teleoperation buyer scenario
A realistic teleoperation request starts when a robotics team has a model behavior that fails in robot workcells, warehouses, kitchens, and labs. The team does not just need more video; it needs examples where timestamp alignment, state/action completeness, and recoverable failure examples can be verified repeatedly [4].
[5]"RoboSet explicitly documents teleoperated trajectories for robot manipulation datasets."
That means the supplier must show the requested robot state, action traces, and synchronized camera streams, prove the capture context, and deliver MCAP, ROS bag, HDF5, RLDS, or LeRobot in a way the buyer can test before scaling.
Teleoperation sample acceptance criteria
A useful sample for teleoperation dataset should include at least one accepted episode, one borderline or failed example, a complete metadata manifest, and a note explaining how the supplier would scale from the sample to 20-200 hours of teleoperated episodes [6]. If the sample cannot show timestamp alignment, state/action completeness, and recoverable failure examples, the buyer should reject it before funding a larger batch.
Related pages
Use these to move from category-level context into specific task, dataset, format, and comparison detail.
External references and source context
- Teleoperation datasets are becoming the highest-intent physical AI content category
ALOHA uses a custom teleoperation interface to collect real demonstrations.
tonyzhaozh.github.io ↩ - Google Research blog
RT-1 is a real robot action-learning reference that depends on observation-action data.
robotics-transformer1.github.io ↩ - Project site
UMI is a portable gripper data collection project for in-the-wild manipulation demonstrations.
umi-gripper.github.io ↩ - Project site
Open X-Embodiment normalizes robot observations and actions across embodiments for policy training.
robotics-transformer-x.github.io ↩ - Dataset page
RoboSet explicitly documents teleoperated trajectories for robot manipulation datasets.
robopen.github.io ↩ - LeRobot GitHub repository
LeRobot provides tooling for recording, converting, and training with robot datasets.
GitHub ↩
FAQ
What is teleoperation dataset?
teleoperation dataset refers to data collected for robot workcells, warehouses, kitchens, and labs. It usually includes robot state, action traces, and synchronized camera streams, metadata, and task outcomes that help train or evaluate physical AI systems.
What should a sourcing request include?
It should include task definition, environment, modality, volume, format, rights, consent, budget, deadline, and QA checks such as timestamp alignment, state/action completeness, and recoverable failure examples.
What format should buyers request?
MCAP, ROS bag, HDF5, RLDS, or LeRobot is the recommended starting point, but truelabel can route buyer-defined schemas when the training pipeline needs a custom layout.
Can this be exclusive?
Yes. Net-new sourcing requests can request exclusive commercial rights, while off-the-shelf datasets are usually non-exclusive unless the buyer explicitly purchases exclusivity.
Sourcing data for teleoperation dataset
Specify the environment, scale, and rights you need. Truelabel matches you with capture partners delivering teleoperation dataset data with consent artifacts and commercial licensing attached.
Request teleoperation training data