Environment · Egocentric data
Egocentric Video Data for Factory & Manufacturing
Factory egocentric video data is first-person, head-mounted footage of manufacturing and assembly work — hands fastening, fitting, inspecting and tending machines from the worker's own point of view. Egocentric-10K gives you 10,000 permissively-licensed raw hours, but every annotated industrial corpus is non-commercial or copyleft, so custom capture is the only clean route to labelled, consented, factory-specific training data.
Quick facts
- Resolution
- 1080p @ 30fps baseline; stereo 2160p @ 60fps for tight-tolerance fitment steps
- Field of view
- ≥120° horizontal — captures the full reach envelope and both hands at a bench
- Mount
- Head-mounted (helmet, cap or glasses rig); never chest-mount or fixed tripod — the camera must follow gaze-directed manipulation
- Sensors
- RGB (baseline), IMU — head, optional wrist, Gaze (optional, for attention on inspection/defect steps), Depth (optional, for tight-tolerance assembly)
- Labels
- Frame-aligned action/step segments (grasp, insert, fasten, gauge, inspect); Object and part-SKU states in frame; Tool-in-use spans (driver, wrench, gauge, fixture)
- Volume
- 40–400 accepted hours per facility/task program; pilot batch in days
Key papers
Hard citations for the claims above. Each entry pairs a specific number with the paper that reports it.
EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video
829 hours, 194 tasks. EgoDex pairs 829 hours of egocentric video across 194 tabletop tasks with 3D hand and finger tracking captured on Apple Vision Pro — the largest and most diverse dexterous human-manipulation dataset to date.
Ego4D: Around the World in 3,000 Hours of Egocentric Video
3,670 hours, 74 locations. Ego4D spans 3,670 hours of daily-life first-person video from 931 camera wearers across 74 locations in 9 countries, collected under consenting-participant privacy and de-identification standards.
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
1,286 hours, 740 participants. Ego-Exo4D pairs simultaneously-captured egocentric and exocentric video of skilled activity from 740 participants across 13 cities — 1,286 hours with multichannel audio, eye gaze, 3D point clouds, camera poses, and IMU.
What factory & manufacturing egocentric data captures
Factory egocentric video data is first-person, head-mounted footage of manufacturing and assembly work — hands fastening, fitting, inspecting and tending machines from the worker's own point of view. Egocentric-10K gives you 10,000 permissively-licensed raw hours, but every annotated industrial corpus is non-commercial or copyleft, so custom capture is the only clean route to labelled, consented, factory-specific training data.
The capture settings this covers:
- Manual assembly-station work: fastening, torqueing, seating and fitting components by hand at a fixed bench or moving line
- Machine tending — loading, unloading and operating presses, CNC cells and injection machines from the operator's viewpoint
- Quality inspection and rework: part-in-hand examination, gauging with calipers, and fixing defects at an inspection station
- Kitting, picking and material handling between stations, including cart and bin interactions
- Maintenance, changeover and repair procedures on production equipment (lockout/tagout, tool swaps, calibration)
- Tool use in frame: power drivers, torque wrenches, jigs, fixtures and hand tools grasped and manipulated at close range
Why robotics and AI labs need factory & manufacturing data
Apple's EgoDex pretrains dexterous manipulation on large-scale egocentric human video, making first-person work footage the base layer of the training stack. [1]
EgoLive shows large-scale egocentric human demonstrations of real-world tasks lifting manipulation policies, so paired human POV data directly improves robot skill. [2]
EgoScale shows dexterous-manipulation performance scaling with the volume and diversity of egocentric human data — the log-linear curve that makes a large factory-specific corpus worth capturing. [3]
AoE shows scalable, low-cost collection of egocentric human video of manual tasks can answer the data scarcity a factory assembly corpus is built to close. [4]
Capture and delivery spec
Every factory & manufacturing capture program runs to an explicit spec so the footage is training-ready on delivery rather than after a re-shoot. The baseline below is tuned per program; sensors, labels, and volume scale with the buyer's model.
| Spec | Detail |
|---|---|
| Resolution | 1080p @ 30fps baseline; stereo 2160p @ 60fps for tight-tolerance fitment steps |
| Frame rate | 30fps baseline, 60fps for fast fastening and insertion motions |
| Field of view | ≥120° horizontal — captures the full reach envelope and both hands at a bench |
| Mount | Head-mounted (helmet, cap or glasses rig); never chest-mount or fixed tripod — the camera must follow gaze-directed manipulation |
| Sensors | RGB (baseline), IMU — head, optional wrist, Gaze (optional, for attention on inspection/defect steps), Depth (optional, for tight-tolerance assembly) |
| Labels | Frame-aligned action/step segments (grasp, insert, fasten, gauge, inspect); Object and part-SKU states in frame; Tool-in-use spans (driver, wrench, gauge, fixture); Assembly-stage / procedure-step index; Optional 21-keypoint 3D hand pose per hand |
| QA gates | Hands-in-frame above threshold on manipulation clips; Stability / no motion blur on the action frames; FOV and horizon check; PPE present and no identifiable bystander faces; Per-clip consent + facility-release artifact attached |
| Delivery | H.265 + per-clip JSON metadata (steps, objects, tools, hand pose), Hugging Face-streamable; consent and facility-release artifacts shipped per clip |
| Volume | 40–400 accepted hours per facility/task program; pilot batch in days |
Open factory & manufacturing datasets
The 5 open corpora most relevant to factory & manufacturing are compared below on scale, sensors, license, commercial use, and the gap each leaves for a buyer. Only 1 of the 5 is permissively licensed for commercial use — which is the whole reason custom capture exists.
| Dataset | Size / scale | Sensors | License | Commercial use | Gap |
|---|---|---|---|---|---|
| Egocentric-10K | 10,000 h · 2,138 workers · 87 factories | Head-mounted RGB (~1080p); no depth/IMU/gaze, no labels | Apache 2.0 | Yes | Raw video only — no action labels, no hand pose, no step segmentation, and an undocumented worker-consent chain. You annotate and legally diligence it yourself. |
| Assembly101 | 101 toy-vehicle (dis)assembly sequences · egocentric + multi-view exo | RGB ego + exo, plus 3D hand poses | Non-commercial research | No | Take-apart toy vehicles, not a real production line; the NC license bars training a shippable product on it. |
| IndustReal | ~6 h · toy-construction assembly procedures | Egocentric RGB + procedure-step labels | CC BY-SA | Conditional | Tiny (~6 h) toy proxy; the ShareAlike copyleft can attach obligations to models trained on it. |
| MECCANO | Toy motorbike assembly · single procedure | RGB + depth + gaze (multimodal) | Research-only | No | One toy-model task with no real-facility variety; research-only terms exclude commercial use. |
| HoloAssist | ~166 h · two-person assembly & repair · HoloLens 2 | RGB + eye/hand tracking + instructor dialog | CDLA-Permissive-style research | Conditional | Staged instructor-student repair sessions, not autonomous production work; confirm license terms before any commercial use. |
Open datasets vs Truelabel custom capture
Licensing reality: of the five corpora above, only Egocentric-10K is commercially usable — and it ships zero labels and no documented consent. Every annotated industrial set (Assembly101, MECCANO, HoloAssist) is non-commercial or research-only, and IndustReal is copyleft. There is effectively no labelled, commercially-clean egocentric factory corpus to buy; custom capture ships a signed consent + facility-release chain on every clip.
Toy-proxy gap: the open annotated sets are toy motorbikes and take-apart vehicles, not your parts, your fixtures, or your station layout. Custom capture matches your exact SKUs, tools and process so the policy learns the task it will actually run.
Taxonomy control: open corpora freeze someone else's action and step schema. Custom capture lets you define the step taxonomy and object set your model predicts, instead of remapping a research benchmark.
Exclusivity and freshness: public datasets are trained on by every competitor and reflect old processes. Custom-capture footage is exclusive, dated, and shot on your current line.
Factory & manufacturing: by the numbers
The figures below are specific to factory & manufacturing egocentric data and anchor the comparisons above.
- Egocentric-10K: 10,000 h · 2,138 workers · 87 factories (the only Apache-2.0 industrial corpus)
- Assembly101: 101 toy-vehicle (dis)assembly sequences, non-commercial
- IndustReal: ~6 h toy-construction corpus under CC BY-SA copyleft
- MECCANO: single toy-motorbike task with RGB + depth + gaze, research-only
- HoloAssist: ~166 h of two-person assembly/repair on HoloLens 2
- TrueLabel factory programs: 40–400 accepted hours per facility, pilot batch in days
How Truelabel captures factory & manufacturing data
Truelabel runs factory & manufacturing programs on a network of 20,000+ consented collectors across nine countries, capturing to your brief on a head-mounted rig. Every clip passes per-clip machine QA — head-mount stability, field of view, and hands-in-frame coverage — and ships with a signed wearer consent artifact and provenance manifest. A calibration pilot returns its first batch in days, then accepted batches scale to 40–400 accepted hours per facility/task program; pilot batch in days, delivered as H.265 + per-clip JSON metadata (steps, objects, tools, hand pose), Hugging Face-streamable; consent and facility-release artifacts shipped per clip. Go deeper via industrial egocentric video sourcing spec, warehouse egocentric video, kitchen egocentric video, what egocentric data is, and egocentric data licensing.
Related pages
Use these to move from category-level context into specific task, dataset, format, and comparison detail.
External references and source context
- EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video
EgoDex is a large-scale egocentric video corpus built to pretrain dexterous manipulation from first-person human video.
arXiv ↩ - EgoLive: A Large-Scale Egocentric Dataset from Real-World Human Tasks
EgoLive is a large-scale egocentric dataset of real-world human tasks used to lift manipulation policies.
arXiv ↩ - EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data
EgoScale supports the claim that dexterous-manipulation performance scales with the volume and diversity of egocentric human data.
arXiv ↩ - AoE: Always-on Egocentric Human Video Collection for Embodied AI
AoE frames scalable, low-cost collection of egocentric human video of manual tasks as an answer to embodied-AI data scarcity.
arXiv ↩ - Humanoid data: 10 Things That Matter in AI Right Now | MIT Technology Review
MIT Technology Review frames the humanoid data bottleneck that makes real-world egocentric capture urgent.
MIT Technology Review - Egocentric-10K dataset card and license
The Egocentric-10K dataset card is the source for the Apache-2.0 license and the 10,000 h / 2,138 workers / 87 factories figures.
Hugging Face - Assembly101: A Large-Scale Multi-View Video Dataset
Assembly101 is the reference for a non-commercial toy-vehicle assembly corpus with 3D hand poses.
Assembly101 project - IndustReal: industrial procedure-execution dataset
IndustReal is the reference for a small CC BY-SA egocentric toy-assembly procedure corpus.
IndustReal project - MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domain
MECCANO is the reference for a research-only multimodal (RGB/depth/gaze) toy-assembly corpus.
University of Catania (IPLAB) - HoloAssist: an Egocentric Human Interaction Dataset
HoloAssist is the reference for staged two-person egocentric assembly/repair captured on HoloLens 2.
Microsoft Research
FAQ
Why not just train on Egocentric-10K — it's free and Apache-2.0?
You can, and it's a strong pretraining base for raw first-person motion. But it has no action labels, no hand pose and no step segmentation, so you carry the full annotation cost, and its worker-consent chain is undocumented, so you carry the legal diligence. It also isn't your factory — no control over the parts, tools or tasks your policy has to imitate.
Is there any annotated, commercially-licensed egocentric factory dataset?
Not really. Assembly101, MECCANO and HoloAssist are the closest annotated industrial corpora and all three are non-commercial or research-only; IndustReal adds procedure labels but is CC BY-SA copyleft. For a policy you plan to ship, custom capture is the only clean path.
Can you match our specific parts, tools and line layout?
Yes — that's the point of custom capture. We brief collectors on your SKU set, fixtures, tools and step taxonomy, then QA every clip against that spec so the footage is your process, not a toy proxy.
How is consent handled for workers and bystanders on a shop floor?
Every wearer signs a capture consent and the facility signs a release; bystanders are handled by framing and blur rules, and each clip ships with a consent artifact. That auditable chain is exactly what the open corpora lack.
Do you capture hand pose and IMU alongside RGB, or just video?
RGB is the baseline. Head and wrist IMU, gaze, depth and 21-keypoint 3D hand pose are optional streams priced per add-on — most assembly buyers take at least head IMU and hand pose so the data is VLA-ready.
Do we get exclusive rights, or is the same footage sold to competitors?
Custom-capture programs are exclusive by default. The footage is yours, dated to your process, and not resold — unlike a public dataset every competitor has already ingested.
Looking for factory egocentric video dataset?
Specify modality, task, environment, rights, and delivery format. Truelabel matches you with vetted capture partners and helps scope consent artifacts and commercial licensing requirements before delivery.
Request factory egocentric data