Teleoperation briefings

Quick facts

Topic

Teleoperation data

Briefings tagged

Procurement-grade analyses

Reference rigs

ALOHA, Mobile ALOHA, SO-100, GELLO

Standard schemas

RLDS, LeRobot, MCAP, ROS bags

Adjacent topics

Bimanual, consent, provenance, licensing

What is teleoperation data?

Teleoperation data is the recorded trace of a human operator piloting a robot through a real-world task. Every frame carries proprioception, action streams, and synchronized camera feeds at 30 Hz or higher — the exact substrate that vision-language-action models need to learn manipulation from demonstration rather than from simulated rollouts ^[1]. RT-1 trained on roughly 130k real-world demonstrations across 13 robots collected over 17 months, and that pattern (volume measured in demonstrations and operator-hours, not images) is the unit teleop briefings use. Briefings filed under this topic focus on which teleop corpora are commercially usable, where buyer-readiness gaps appear, and how operator skill, embodiment, and acceptance criteria change the value of each session.

Procurement decisions turn on details public dataset cards rarely surface: operator identity and consent, robot embodiment, action-space schema, camera sync tolerance, and accepted failure modes. The truelabel briefings explain why a benchmark dataset can be the wrong baseline for a frontier policy and when a buyer should commission net-new collection against a strict capture spec, routed through the truelabel marketplace via a written sourcing brief.

The category sits at the centre of the 2026 physical-AI supply chain. Foundation-model teams want it because demonstrations carry action labels that pure egocentric video does not ^[2]. Humanoid teams want it because embodiment-matched traces are the only data that survives sim-to-real transfer. Warehousing, kitchen, and surgical-assist programs want it because their task distributions — often fewer than 50 named SKUs or 20 procedural variants — are too narrow for any public benchmark to bootstrap.

What to ask a teleop supplier before a purchase order

Procurement teams that treat teleoperation like generic video annotation lose the deal twice — once on yield, once on rights. The supplier conversation should open on embodiment and close on consent, with everything in between scoped to the action-space schema the buyer's policy actually consumes ^[3]. Start with the rig: ALOHA, ALOHA-Pro, Mobile ALOHA, SO-100, GELLO, custom dual-arm, or humanoid teleop chair. Each rig has a different action distribution and a different operator pool. A buyer training on a UR5 in production should not accept Franka Panda traces as a substitute unless the policy is explicitly cross-embodiment.

Next, the action-space schema. RLDS, LeRobot, and MCAP all serialise teleop differently, and within each format the action vector can be joint position, joint velocity, end-effector pose delta, or absolute pose. The buyer's training loop assumes one of those; the supplier needs to ship the same one. Ask for a 60-second sample trajectory in the target schema before signing anything larger than a pilot. If the supplier cannot produce a sample in the buyer's format inside a week, the rest of the engagement will be worse.

Then sync tolerance. Cameras, proprioception, and action commands must land on a shared clock with drift bounded in milliseconds, not frames. The accepted bar for bimanual policies is sub-frame drift across left arm, right arm, and at least two camera angles ^[4]. Ask the supplier how they verify this — hardware-triggered capture, ROS bag timestamping, MCAP frame indexing — and ask for the verification artifact, not just the claim.

Finally, the rights stack. Operator identity, consent scoped to commercial training, redistribution rights, and derived-model rights are the four artifacts a procurement memo will be measured against twelve months later.

What's inside a teleop session?

A teleop session is not a video. It is a synchronized multimodal log running at 30-60 Hz on the camera streams and 100-500 Hz on the proprioception streams: one or more RGB or RGB-D camera streams, robot proprioception (joint positions, velocities, torques), end-effector pose, gripper or hand state, and the operator's action commands, with sub-10 ms drift tolerance across streams ^[5]. The session also carries a task spec, the embodiment identifier, the operator's skill tier, and the accepted failure modes for that task. When a briefing under this topic describes a dataset as procurement-grade, it means all of those fields are present, aligned, and addressable from a single index.

The serialization layer matters because it determines what a training pipeline can consume without a custom loader. RLDS is the Google-led TFDS-compatible format that anchors Open X-Embodiment — Open X-Embodiment aggregates roughly 1.4 million trajectories across 22 institutional sources under a shared schema. LeRobot is Hugging Face's PyTorch-first equivalent, optimised for imitation learning loops. MCAP is the foundation-format that ROS-native pipelines tend to land on because it preserves message-level fidelity; see the datasets format guide for the side-by-side.

Rig identity drives both the action space and the operator economics. ALOHA's leader-follower bimanual setup costs roughly $20,000 in parts and produces high-fidelity dual-arm traces at 50 Hz across 2x 6-DOF + grippers ^[3]. The SO-100, GELLO, and similar low-cost designs (under $1,000) widen the operator pool by reducing the capital barrier. Humanoid teleop chairs push the action space into upper-body dexterity but narrow the operator pool sharply.

The buyer-relevant technical statement is never that we have teleop data. It is that we have RLDS-schema bimanual traces on an ALOHA-class rig with sub-frame sync across two camera angles and joint-position action vectors, captured by Tier-2 operators against a written accept rule. That sentence is the unit of comparison.

Why does teleop procurement fail?

The dominant failure mode is embodiment mismatch. A buyer pretraining on Open X-Embodiment, then fine-tuning on whichever bimanual corpus is cheapest, then deploying on a custom dual-arm rig will discover at evaluation time that the policy has learned the source rig's kinematics ^[6]. The fix is not more data — it is embodiment-matched data captured against the deployment rig's action space, typically 1,000-10,000 trajectories on the deployment rig.

The second failure mode is yield collapse under an unstated accept rule. Suppliers price teleop by session, hour, or trajectory. Buyers consume it by accepted trajectory, which is a different number. A session can look complete on delivery and still produce a 30-40% accept rate against a strict spec — wrong gripper closure timing, missed contact events, drifted sync above the 10 ms tolerance, operator-side procedural errors. Without an accept rule written into the spec and enforced at sample-packet review, the buyer pays for the session and absorbs the 30-40% yield gap.

The third failure mode is consent that does not survive a deployment review. An operator who agreed to research use is not an operator who agreed to commercial model training, and a model trained on the former cannot be shipped on the latter. This is the failure mode that surfaces 12-18 months into a programme, when legal review intercepts the production deployment. The structural fix is to write consent scope into the teleop spec at capture time and link it through the provenance chain, not retrofit it from a dataset card.

"Overall we have 30,050 trajectories in the dataset, out of which 9,500 are collected through teleoperation."

That public-dataset pattern is the minimum bar for marketplace intake: ask for trajectory counts, camera viewpoints, task and scene coverage, and failure labels before funding scale-up.

Workflow: from sourcing request to accepted trajectory

Every truelabel teleop sourcing request follows a sequence. The four steps below are the operational template the briefings under this topic reference repeatedly. None of them should be skipped, and each produces an artifact a deployment review can audit later ^[8]. The same sequence applies whether the buyer is sourcing 1,000 trajectories or 100,000 — the difference is volume, not order.

The sequence collapses neatly into a four-step process because the procurement-grade questions stack: rig fit gates schema fit, schema fit gates sync verification, sync verification gates the accept rule, and the accept rule gates scale-up. Skipping a step compresses the gap between sample acceptance and deployment-time failure into a costly window.

1. 01

   Specify embodiment and operator tier

   Name the deployment rig, the action-space schema, and the operator tier required for the task. Reject suppliers who cannot match all three before any other diligence runs.

2. 02

   Demand a sample trajectory in target schema

   Ask for a 60-second sample in RLDS, LeRobot, or MCAP per the buyer's training loop. The sample exercises sync, schema, and operator skill in one delivery before a contract is signed.

3. 03

   Verify sync and accept-rule criteria

   Run the sample through the buyer's ingestion pipeline. Confirm sub-frame drift across arms and cameras, joint-position action continuity, and contact-event labelling. Document the accept rule in writing.

4. 04

   Scale capture against the written spec

   Only after sample acceptance does volume capture begin. Each delivery is reviewed against the accept rule; consent records and per-trajectory metadata ship with the data, not as a follow-up.

Why teleop sits at the centre of physical-AI demand

Commercial humanoid teams are pursuing real-world teleoperation data from deployment environments rather than relying on public benchmarks alone ^[9]. The structural reason is the same one that pushes Scale AI's physical-AI data engine toward custom collection: the buyer's task distribution is too narrow for any public corpus to bootstrap.

Two-handed teleop demands operator training and coordination skill above what single-arm capture requires; yield per operator-hour is lower, and the operator pool is smaller, especially for skill-tier-3 tasks like assembly or assisted-surgical procedures. Briefings under this topic name the operator-skill tier explicitly because two suppliers with the same hourly rate can deliver very different procurement value once the accept rate is in.

The supply-side rigs themselves drive the briefings. ALOHA and Mobile ALOHA dominate published bimanual work; the SO-100 and GELLO open the operator pool by lowering rig cost; humanoid teleop chairs and exoskeleton-style interfaces are pushing into upper-body dexterity. Each rig produces a different action-space schema and a different yield curve.

How teleop composes with adjacent truelabel topics

Teleoperation rarely lives in isolation. A briefing tagged teleoperation almost always carries a secondary tag covering embodiment provenance, commercial-use posture, or consent artifacts. The procurement memo a buyer writes about teleop data is the same memo that closes commercial-use and consent risk, because the three artifacts compose into the deployment review.

A common pattern in the briefings: a corpus is technically sound (good rig, good schema, good sync) but underbuilt on consent or provenance, which lifts the buyer-readiness score from procurement-grade to research-only. Briefings flag the composition explicitly so a reader does not score a corpus on one axis and skip the other two.

The cross-link to VLA training literature is also load-bearing. The same trajectory format that supports research-grade benchmarking does not always survive a commercial training run — the action distribution, the operator tier, and the embodiment match all show up in evaluation metrics months after acquisition. Briefings under this topic write the cross-link out so a buyer evaluating teleop against a VLA training plan can audit both at once.

Briefing index and recurring patterns

Briefings tagged teleoperation share a recurring structural shape. Each item names a corpus or supplier, an embodiment, the action-space schema, the consent posture, and the buyer implication in one sentence. The recurring pattern lets a procurement reader scan an entire archive in twenty minutes and exit with a short list of corpora worth profiling further.

Expect this topic to grow faster than dataset-licensing or egocentric-video coverage through 2026. Teleoperation is where commercial physical-AI spend is concentrating, and it is the topic that pulls the rest of the truelabel taxonomy — consent, provenance, commercial-use, bimanual-manipulation — into every sourcing memo. Use this topic as a starting point when scoping any policy-learning program; pair it with consent for the rights stack and with bimanual-manipulation when the deployment task involves coordinated two-arm work.

Practical patterns: how a buyer uses teleoperation briefings in a sourcing memo

Procurement memos cite briefings for a reason: the briefings carry the source evidence the memo cannot reconstruct from a vendor pitch deck. A memo that names teleoperation as the load-bearing variable should quote the briefings that profile the candidate sources, copy the buyer-implication sentence verbatim, and date-stamp the citation so a re-audit cadence can be set against the freshness of the brief ^[3].

The first practical pattern is sequencing: scan the topic archive before any supplier outreach, narrow to two or three candidate sources, then enter supplier conversations with the briefing's buyer-implication sentence as the opening question. Suppliers who have read the same briefings tend to respond faster and more substantively because they can see the gap the buyer is trying to close. Suppliers who have not read them tend to pitch their default offering, which is usually a poor match for a topic-specific sourcing request.

The second pattern is composition. A briefing under teleoperation rarely lives alone — it almost always carries a secondary tag covering one of the procurement layers (consent, licensing, commercial-use, provenance). A memo that quotes any teleoperation briefing should also quote the corresponding briefing under the secondary tag, so the procurement question is answered across both layers rather than only the primary one ^[10].

The third pattern is the buyer-implication chain. Each briefing's buyer-implication sentence becomes a memo line; each memo line becomes a supplier question; each supplier question becomes a contract clause; each contract clause becomes a delivery-acceptance check. A briefings archive used this way is not a reading list — it is the procurement workflow with citations attached workflow guidance.

What good looks like across teleoperation briefings

Across the teleoperation archive, the briefings that survive a deployment review six months later share a pattern. They name the source with version, they cite the rights and consent posture inside the source (not the dataset card), they identify the embodiment or capture rig explicitly, they date-stamp the review, and they end with one sentence a procurement memo can quote without modification. The pattern is shorter than the typical research write-up because the audience is different — a procurement reader does not need the lit review, they need the buyer implication.

A good briefing also names what is missing. The hardest part of writing a buyer-grade brief is admitting that a candidate source does not clear the bar for the deployment context. Briefings under teleoperation that name the gap explicitly are more useful than briefings that paper over it, because the procurement memo has to cite the gap to defend the decision to commission custom capture instead via the marketplace.

The third quality marker is freshness. Robotics datasets, vendor positions, and capture rigs move quickly. A briefing that is six months old needs a freshness header that says so; a briefing that has been re-audited and confirms the original position needs a date-stamp on the re-audit. Briefings under teleoperation that maintain this freshness cadence are the ones procurement teams cite repeatedly across multiple sourcing engagements.

The fourth quality marker is cross-link discipline. A briefing that closes by naming the adjacent topics it depends on (consent, licensing, provenance, embodiment, capture rig) gives the reader the entry point into the rest of the archive. Briefings under teleoperation that do this consistently let a procurement reader navigate the archive as a working surface rather than a flat list of articles.

Reading teleoperation briefings as a working file, not a static archive

The briefings under this topic are designed to be a working file. The archive is not a textbook; it is a procurement reference whose entries are written once, re-audited on cadence, and discarded when the underlying source changes in a way that invalidates the original brief. A buyer who treats the archive as a working file gets value from it every quarter; a buyer who treats it as a static archive reads it once and never returns.

Use the archive in three modes. In sourcing-decision mode, scan the topic, narrow to two or three candidates, and enter supplier conversations with the buyer-implication sentence as the opening question. In re-audit mode, revisit the briefings whose sources have changed (publisher term updates, contributor withdrawals, new releases) and update the procurement memos that cite them. In planning mode, read the topic archive end to end to build a mental model of where the buyer-readiness gaps cluster and what the dominant recommendation patterns look like.

The fourth use case is briefing-to-briefing comparison. A buyer reading two briefings under teleoperation side by side can compare the buyer-implication sentences directly because the briefings follow the same structural shape. The comparison is the lightest-weight diligence step in the workflow and the most common reason to enter the archive in the first place. Briefings under teleoperation are written to support this comparison: same shape, same fields, different sources ^[3].

A working archive also needs an entry point and an exit point. The entry point is this topic page, with its TL;DR, sample-spec quick-facts, comparison table, and steps block. The exit point is the briefing card whose buyer implication a procurement memo cites. Everything between is the reading workflow the briefings are designed to support.

Common mistakes when buyers ignore teleoperation

The dominant mistake when teleoperation is treated as a secondary concern is sequencing: the buyer commits to a source on the basis of the catalog presence, the licence label, or the supplier pitch, and discovers the teleoperation-related gap weeks or months later when the policy is already partway through training. The cost of that mistake is retraining cost plus schedule cost; the structural fix is to treat teleoperation as a gating field before training compute, not after ^[3].

The second mistake is partial coverage. A corpus that scores well on teleoperation for 80% of trajectories and poorly for 20% is not 80% usable — it is unusable for any pipeline that cannot filter at the trajectory level. The briefings under this topic flag partial-coverage candidates explicitly because the gap is structural and the fix is rarely available downstream. The procurement-grade pattern is to require complete coverage at the spec level or to plan for the surgical removal of the non-compliant fraction before training starts.

The third mistake is reliance on aggregator labels. Aggregators pool sources under a single banner and a single posture, but the upstream chain frequently breaks at the second or third hop ^[10]. A buyer using an aggregator-licensed corpus needs to verify that every upstream source supports the aggregator's release terms; aggregators rarely surface this verification, so the buyer carries the diligence cost. Briefings under teleoperation flag aggregator-inherited risk for the cases where the inheritance chain is most likely to break.

The fourth mistake is treating the topic as resolved when only the label has been checked. teleoperation is an engineering and contractual problem; resolving it requires evidence (sample artifacts, audit trails, per-trajectory metadata) rather than assertion. Suppliers who can produce evidence are procurement-grade; suppliers who can only assert are research baselines. The briefings under this topic name the evidence explicitly so the buyer can distinguish between the two.

Related pages

Use these to move from category-level context into specific task, dataset, format, and comparison detail.

External references and source context

1. RT-1: Robotics Transformer for Real-World Control at Scale

   RT-1 demonstrates that discretized action tokens over real-robot demonstrations train a generalist manipulation policy.

   arXiv ↩
2. OpenVLA: An Open-Source Vision-Language-Action Model

   OpenVLA fine-tunes Llama-2 7B on Open X-Embodiment trajectories and matches RT-2-X performance at lower parameter count.

   arXiv ↩
3. Teleoperation datasets are becoming the highest-intent physical AI content category

   ALOHA leader-follower bimanual setup produces high-fidelity dual-arm traces at modest hardware cost, which is why so much published bimanual work cites it.

   tonyzhaozh.github.io ↩
4. Project site

   DROID demonstrates real-world behavior cloning with synchronized observations and robot actions across diverse scenes and embodiments.

   droid-dataset.github.io ↩
5. MCAP file format

   MCAP stores timestamped multimodal robotics logs at message-level fidelity for replay and audit.

   mcap.dev ↩
6. Project site

   Open X-Embodiment aggregates trajectories across many robots and tasks under a shared schema; cross-embodiment pretraining is the dominant use.

   robotics-transformer-x.github.io ↩
7. Dataset page

   RoboSet reports 9,500 teleoperated trajectories collected as part of a 30,050-trajectory release.

   robopen.github.io ↩
8. truelabel physical AI data marketplace bounty intake

   Truelabel routes teleop sourcing requests to vetted capture partners with sample-review before scale-up.

   truelabel.ai ↩
9. Figure + Brookfield humanoid pretraining dataset partnership

   Figure AI and Brookfield announced a partnership to collect humanoid teleoperation data in deployment environments.

   figure.ai ↩
10. RLDS: Reinforcement Learning Datasets

    RLDS defines episode-level robot-learning datasets with observations, actions, rewards, discounts, and step metadata.

    GitHub ↩

FAQ