Industry · Egocentric data
Egocentric Video Data for Construction
A construction site dataset for egocentric AI is first-person, head-mounted footage of trade work — framing, electrical rough-in, finishing, concrete and material handling — shot from the worker's own point of view, with PPE and tools in frame. No commercially-licensed egocentric construction corpus exists as of 2026: the nearest open sources are worn-camera industrial footage and staged assistive-task video, none of it on an active jobsite. Consented on-site custom capture against a trade taxonomy is the only route to training-ready data.
Quick facts
- Resolution
- 1080p @ 30fps baseline; 2160p for fine finishing and PPE-detail frames
- Field of view
- ≥120° horizontal — holds both hands, the tool tip and the immediate hazard context at arm's length
- Mount
- Hard-hat or head-strap rig integrated with required PPE; never chest-mount or fixed tripod — the camera has to follow gaze-directed trade work
- Sensors
- RGB (baseline), IMU — head, optional wrist for tool motion, GPS / site-context tag (optional, for scope and location), Depth (optional, for reach and clearance work)
- Labels
- Trade / scope segments (framing, electrical, finishing, concrete, material handling); Task and step index within a scope (measure, cut, fasten, make-up, float); PPE-compliance flags (hard hat, hi-vis, gloves, eye protection) per frame
- Volume
- 40–300 accepted hours per trade / site program; calibration pilot batch in days
Key papers
Hard citations for the claims above. Each entry pairs a specific number with the paper that reports it.
HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World
166 hours, 350 pairs. HoloAssist is a large-scale egocentric human-interaction dataset — 166 hours captured by 350 unique instructor-performer pairs on a mixed-reality headset with seven synchronized data streams — built for interactive AI assistants that guide people through real-world physical tasks.
EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video
829 hours, 194 tasks. EgoDex pairs 829 hours of egocentric video across 194 tabletop tasks with 3D hand and finger tracking captured on Apple Vision Pro — the largest and most diverse dexterous human-manipulation dataset to date.
Ego4D: Around the World in 3,000 Hours of Egocentric Video
3,670 hours, 74 locations. Ego4D spans 3,670 hours of daily-life first-person video from 931 camera wearers across 74 locations in 9 countries, collected under consenting-participant privacy and de-identification standards.
What construction needs from egocentric data
A construction site dataset for egocentric AI is first-person, head-mounted footage of trade work — framing, electrical rough-in, finishing, concrete and material handling — shot from the worker's own point of view, with PPE and tools in frame. No commercially-licensed egocentric construction corpus exists as of 2026: the nearest open sources are worn-camera industrial footage and staged assistive-task video, none of it on an active jobsite. Consented on-site custom capture against a trade taxonomy is the only route to training-ready data.
The capture settings this covers:
- Framing and carpentry: a carpenter's hands measuring, marking, cutting and nailing studs and sheathing, with the tape, square and nail gun moving through the frame.
- Electrical rough-in: pulling and stripping wire, mounting boxes, making up devices in a panel — close-range hand work against exposed structure.
- Finishing trades: hanging and taping drywall, setting tile, trimming and painting, where the surface and the tool tip are what the camera has to hold.
- Concrete and masonry: screeding, floating and trowelling wet pours, laying block, mixing — heavy two-handed work with the material state changing shot to shot.
- Material handling and rigging between stations: carrying lumber, loading a hoist, moving a pallet, signalling a lift — whole-body movement across an uneven site.
- Safety-relevant frames throughout: hard hat and hi-vis on the wearer, gloves and eye protection on the hands, ladders and edges in view — the PPE and hazard context a safety model has to read.
Why construction needs first-person human video
Apple's EgoDex pretrains dexterous manipulation on large-scale egocentric human video, so a construction worker's first-person view of tool and material handling sits in the same training substrate a job-site robot policy learns from. [1]
EgoScale showed dexterous-manipulation performance scaling log-linearly with the volume and diversity of egocentric human data — the curve that makes capturing a large, jobsite-specific trade corpus a measurable return rather than a nice-to-have. [2]
EgoLive shows large-scale egocentric human demonstrations of real-world tasks lifting manipulation policies, so consented head-mounted footage of construction work directly improves the robot skill it targets. [3]
AoE frames scalable, low-cost collection of egocentric human video of manual tasks as an answer to the data scarcity a construction-site capture program is built to close. [4]
Capture and delivery spec
Every construction capture program runs to an explicit spec so the footage is training-ready on delivery rather than after a re-shoot. The baseline below is tuned per program; sensors, labels, and volume scale with the buyer's model.
| Spec | Detail |
|---|---|
| Resolution | 1080p @ 30fps baseline; 2160p for fine finishing and PPE-detail frames |
| Frame rate | 30fps baseline, 60fps for fast tool motion (nailing, cutting, trowelling) |
| Field of view | ≥120° horizontal — holds both hands, the tool tip and the immediate hazard context at arm's length |
| Mount | Hard-hat or head-strap rig integrated with required PPE; never chest-mount or fixed tripod — the camera has to follow gaze-directed trade work |
| Sensors | RGB (baseline), IMU — head, optional wrist for tool motion, GPS / site-context tag (optional, for scope and location), Depth (optional, for reach and clearance work) |
| Labels | Trade / scope segments (framing, electrical, finishing, concrete, material handling); Task and step index within a scope (measure, cut, fasten, make-up, float); PPE-compliance flags (hard hat, hi-vis, gloves, eye protection) per frame; Safety-event and hazard spans (working at height, edge exposure, missing PPE); Tool-in-use spans (nail gun, drill, saw, trowel, level) |
| QA gates | Hands and tools in frame above threshold on task clips; Required PPE present on the wearer and no identifiable bystander faces; Stability / no motion blur on the action frames despite site movement; Per-clip wearer consent + signed site-access release attached |
| Delivery | H.265 clips + per-clip JSON metadata (scope, steps, PPE flags, tools), Hugging Face-streamable; wearer consent and site-release artifacts shipped per clip |
| Volume | 40–300 accepted hours per trade / site program; calibration pilot batch in days |
Open construction datasets
The 4 open corpora most relevant to construction are compared below on scale, sensors, license, commercial use, and the gap each leaves for a buyer. Only 1 of the 4 is permissively licensed for commercial use — which is the whole reason custom capture exists.
| Dataset | Size / scale | Sensors | License | Commercial use | Gap |
|---|---|---|---|---|---|
| Egocentric-10K | 10,000 h · 2,138 workers · 87 facilities | Head-mounted RGB (~1080p); no depth/IMU/gaze, no labels | Apache 2.0 | Yes | The one commercially-usable anchor — but it is worn-camera work inside industrial facilities, not an active construction site: no trade scopes, no PPE or safety labels, and an undocumented worker-consent chain you diligence yourself. |
| Ego4D (jobsite-adjacent slices) | Small outdoor / manual-labour subset of the 3,670 h corpus | Head-mounted RGB (varies by device across contributors) | Ego4D signed license | Conditional | A handful of outdoor and manual-work clips brush against jobsite activity, but nothing is scoped to trades, and the signed license constrains commercial training — you cannot filter it into a clean construction set. |
| HoloAssist | ~166 h · two-person assembly & repair · HoloLens 2 | RGB + eye/hand tracking + instructor dialog | CDLA-Permissive-style research | Conditional | The most permissive license in the set and genuinely task-structured, but the tasks are staged indoor assembly and repair, not framing, wiring or finishing on a live site; confirm terms before any commercial use. |
| IndustReal | ~6 h · toy-construction assembly procedures | Egocentric RGB + procedure-step labels | CC BY-SA | Conditional | Named for construction but it is a six-hour toy-assembly proxy on a bench; the ShareAlike copyleft can attach obligations to any model trained on it, and the contact distribution is nothing like real trade work. |
Open datasets vs Truelabel custom capture
There is no jobsite corpus to buy. Egocentric-10K is the only commercially-clean option and it is worn-camera factory-floor work with zero labels and an undocumented consent chain; Ego4D's manual-labour slices sit under a signed license and are never scoped to trades; HoloAssist and IndustReal are staged indoor assembly, one of them a toy-construction bench proxy. Custom capture is the only path to labelled, consented, on-site trade footage — and every clip ships a signed wearer consent plus a site-access release.
Legal site access is the real moat, not the camera. Filming on an active site means insurer sign-off, a general-contractor release, badging, and a hard-hat mount that keeps its rating. That is exactly why no open corpus exists and why scraped YouTube jobsite video is a liability — no consent, no site release, no right to train. We clear access and mounting up front so the footage is defensible.
Trade taxonomy is yours to define. Open sets freeze someone else's schema, or have none at all. Custom capture lets you specify the scopes — framing, electrical, finishing, concrete, rigging — and the PPE-compliance and safety-event labels your model actually predicts, delivered in your episode format instead of remapped from a research benchmark.
PPE and safety context is captured deliberately, not incidentally. Because it is first-person, the hard hat, hi-vis, gloves and the edge or ladder the worker is standing on are all in the frame the model needs — labelled per clip, rather than hoped for in found footage.
Construction: by the numbers
The figures below are specific to construction egocentric data and anchor the comparisons above.
- No commercially-licensed egocentric construction corpus exists as of 2026-07 — the table profiles the nearest worn-camera and staged-assembly adjacencies and says so
- Egocentric-10K: 10,000 h · 2,138 workers · 87 facilities (Apache-2.0) — the only commercially-clean anchor, and it is factory-floor, not jobsite
- Search demand: "construction site dataset" and "PPE detection dataset" each ~10/mo (DataForSEO, 2026-07-06)
- IndustReal: ~6 h of toy-construction bench assembly under CC BY-SA copyleft — the only open set with 'construction' in its name, and it is a proxy
- HoloAssist: ~166 h of staged two-person assembly/repair on HoloLens 2, the most permissive license in the adjacency set
- TrueLabel construction programs: 40–300 accepted hours per trade/site, with a signed wearer consent plus site-access release on every clip
How Truelabel captures construction data
Truelabel runs construction programs on a network of 20,000+ consented collectors across nine countries, capturing to your brief on a head-mounted rig. Every clip passes per-clip machine QA — head-mount stability, field of view, and hands-in-frame coverage — and ships with a signed wearer consent artifact and provenance manifest. A calibration pilot returns its first batch in days, then accepted batches scale to 40–300 accepted hours per trade / site program; calibration pilot batch in days, delivered as H.265 clips + per-clip JSON metadata (scope, steps, PPE flags, tools), Hugging Face-streamable; wearer consent and site-release artifacts shipped per clip. Go deeper via what egocentric data is, egocentric data licensing, industrial egocentric video sourcing, per-clip consent artifacts, humanoid robot training data, and physical AI data marketplace.
Related pages
Use these to move from category-level context into specific task, dataset, format, and comparison detail.
External references and source context
- EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video
EgoDex is a large-scale egocentric video corpus built to pretrain dexterous manipulation from first-person human video.
arXiv ↩ - EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data
EgoScale reports dexterous-manipulation performance scaling log-linearly with the volume and diversity of egocentric human data.
arXiv ↩ - EgoLive: A Large-Scale Egocentric Dataset from Real-World Human Tasks
EgoLive is a large-scale egocentric dataset of real-world human tasks used to lift manipulation policies.
arXiv ↩ - AoE: Always-on Egocentric Human Video Collection for Embodied AI
AoE frames scalable, low-cost collection of egocentric human video of manual tasks as an answer to embodied-AI data scarcity.
arXiv ↩ - Egocentric video remains useful but incomplete for robot data buyers
Ego4D is a large egocentric corpus under a signed license whose manual-labour and outdoor slices are the nearest jobsite-adjacent footage, but it is not scoped to construction trades.
ego4d-data.org - HoloAssist: an Egocentric Human Interaction Dataset
HoloAssist captures staged two-person assembly and repair on HoloLens 2 under a CDLA-Permissive-style research license.
Microsoft Research - IndustReal: industrial procedure-execution dataset
IndustReal is a ~6-hour egocentric toy-construction assembly dataset with procedure-step labels under CC BY-SA.
IndustReal project
FAQ
Is there an open construction site dataset I can just train on?
Not an egocentric one. The nearest commercially-usable source is Egocentric-10K, and that is worn-camera work inside factories with no trade scopes, no PPE labels and an undocumented consent chain. Ego4D has a few outdoor manual-work clips under a signed license, and HoloAssist and IndustReal are staged indoor assembly — one a toy-construction bench proxy. For a jobsite model you plan to ship, custom capture is the only clean path.
Can collectors legally film on an active site — access, hard-hat mounting, insurer constraints?
Yes, and that access is the point of the program. We handle the general-contractor release, insurer and badging requirements, and mount the camera on a hard hat in a way that preserves its rating. Every clip then ships with the wearer's consent and a signed site-access release — the exact chain scraped jobsite video can never provide.
Do you label PPE compliance and safety events?
Yes. Per-frame PPE flags — hard hat, hi-vis, gloves, eye protection — plus hazard and safety-event spans like working at height, edge exposure or missing PPE, are a standard label layer. Because the footage is first-person, that safety context is actually in frame rather than inferred from a distant fixed camera.
Can you match our trade taxonomy — framing, electrical, finishing, concrete?
That is the core of custom capture. You give us the scopes, tasks and tools your model has to recognise, and we brief collectors and QA every clip against that spec, so the data is your trades and your steps, not a toy-assembly proxy with a fixed research label set.
Why not just scrape YouTube jobsite videos?
Because you would have no right to train on it. Found footage carries no wearer consent, no site release and no commercial licence, which is a liability for a shipping product — and it is rarely head-mounted, so the hands and tools your model needs drop out of frame. Custom capture gives you head-mounted framing plus an auditable consent and site-access chain on every clip.
How much construction egocentric data does a jobsite model actually need?
It scales with the model, and it pays off predictably. EgoScale found dexterous-manipulation performance rising log-linearly with egocentric human-data volume and diversity, and the humanoid field's headline constraint is a shortage of exactly this real-world manual-task video. Most programs open with a 40–300 accepted-hour pilot on one or two trades, then scale once the model responds.
Looking for construction site dataset?
Specify modality, task, environment, rights, and delivery format. Truelabel matches you with vetted capture partners and helps scope consent artifacts and commercial licensing requirements before delivery.
Request construction egocentric data