Environment · Egocentric data
Egocentric Data for Activities of Daily Living (ADL)
An ADL dataset is first-person video of routine household activities — cooking, cleaning, grooming, eating, dressing — captured from a head- or chest-mounted camera so a model sees the hands, objects, and task order the way a home robot must. Every open ADL corpus is dated, scripted, tiny, or non-commercially licensed, so labs training home humanoids and health-monitoring models commission custom capture with a documented consent chain.
Quick facts
- Resolution
- 1080p baseline; stereo 2160p@60 for depth and 3D hand-pose add-ons
- Field of view
- ≥120° horizontal so both hands and the work surface stay in frame during close tasks
- Mount
- Head-mounted (cap or glasses rig), not chest-mounted — chest mounts crop the hands during close manipulation, the flaw that limits the 2012 UCI ADL corpus
- Sensors
- RGB, +IMU (head motion), +gaze (optional), +depth/stereo (optional, for 3D hand-object pose)
- Labels
- frame-aligned ADL action segments (verb + object); object and appliance state changes (open/closed, on/off, empty/full); room/context tag
- Volume
- 40–200 accepted hours per program
Key papers
Hard citations for the claims above. Each entry pairs a specific number with the paper that reports it.
Ego4D: Around the World in 3,000 Hours of Egocentric Video
3,670 hours, 74 locations. Ego4D spans 3,670 hours of daily-life first-person video from 931 camera wearers across 74 locations in 9 countries, collected under consenting-participant privacy and de-identification standards.
Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100
90K action segments on 100 hours. EPIC-KITCHENS-100 densely annotates 100 hours (20M frames) with roughly 90,000 action segments across 45 kitchens — the labeled-hour density that raw first-person corpora cannot match.
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
1,286 hours, 740 participants. Ego-Exo4D pairs simultaneously-captured egocentric and exocentric video of skilled activity from 740 participants across 13 cities — 1,286 hours with multichannel audio, eye gaze, 3D point clouds, camera poses, and IMU.
What home & activities of daily living (ADL) egocentric data captures
An ADL dataset is first-person video of routine household activities — cooking, cleaning, grooming, eating, dressing — captured from a head- or chest-mounted camera so a model sees the hands, objects, and task order the way a home robot must. Every open ADL corpus is dated, scripted, tiny, or non-commercially licensed, so labs training home humanoids and health-monitoring models commission custom capture with a documented consent chain.
The capture settings this covers:
- Meal prep and clean-up: opening the fridge, chopping, using the stove, plating, then loading the dishwasher and wiping the counter
- Cleaning routines: vacuuming, mopping, making a bed, folding and putting away laundry
- Personal care at the bathroom sink: brushing teeth, shaving, applying skincare, taking medication from a pill organizer
- Eating and drinking at a table: pouring, using utensils, refilling a glass, clearing plates
- Tidying and organizing: putting away groceries, sorting mail, watering plants, charging devices
- Assisted-living mobility routines: getting dressed, moving between rooms with a walker, reaching into low and high cabinets
Why robotics and AI labs need home & activities of daily living (ADL) data
EgoScale shows dexterous-manipulation performance scaling with the volume and diversity of egocentric human demonstrations; for a home robot, human ADL footage is the cheapest way to widen skill coverage [1]
The AoE line of work names the shortage of real-world task data the bottleneck holding back humanoid robots, and household chores are exactly the long-tail behavior that simulators and factory teleoperation never capture [2]
Apple's EgoDex treats large-scale egocentric human video as the pretraining substrate for general-purpose manipulation rather than an afterthought [3]
Capture and delivery spec
Every home & activities of daily living (ADL) capture program runs to an explicit spec so the footage is training-ready on delivery rather than after a re-shoot. The baseline below is tuned per program; sensors, labels, and volume scale with the buyer's model.
| Spec | Detail |
|---|---|
| Resolution | 1080p baseline; stereo 2160p@60 for depth and 3D hand-pose add-ons |
| Frame rate | 30 fps standard; 60 fps for fast manipulation like whisking, scrubbing, or wiping |
| Field of view | ≥120° horizontal so both hands and the work surface stay in frame during close tasks |
| Mount | Head-mounted (cap or glasses rig), not chest-mounted — chest mounts crop the hands during close manipulation, the flaw that limits the 2012 UCI ADL corpus |
| Sensors | RGB, +IMU (head motion), +gaze (optional), +depth/stereo (optional, for 3D hand-object pose) |
| Labels | frame-aligned ADL action segments (verb + object); object and appliance state changes (open/closed, on/off, empty/full); room/context tag; optional 25-joint-per-hand pose; optional narration transcript |
| QA gates | hands-in-frame above threshold; exposure and stability check; unscripted-behavior review (reject acted/staged runs); PII and bystander blur pass (faces, screens, documents, mail); consent artifact attached per clip |
| Delivery | H.265 clips + per-clip JSON (actions, objects, appliance states, room, device, consent id); Hugging Face-streamable |
| Volume | 40–200 accepted hours per program |
Open home & activities of daily living (ADL) datasets
The 5 open corpora most relevant to home & activities of daily living (ADL) are compared below on scale, sensors, license, commercial use, and the gap each leaves for a buyer. None of them are cleanly licensed for commercial model training — which is the whole reason custom capture exists.
| Dataset | Size / scale | Sensors | License | Commercial use | Gap |
|---|---|---|---|---|---|
| ADL (UCI, 2012) | ~1M frames · 18 ADL actions · 20 people in their own homes | Chest-mounted RGB, SD-era | Not clearly stated | Conditional | License never clearly stated (treat commercial use as negotiable); chest mount crops the hands; 2012 resolution; behavior is largely acted. |
| Charades-Ego | 7,860 videos · 157 action classes · paired first/third-person | Worn/handheld RGB | Non-commercial (research) | No | Videos are scripted from prompts, not natural home behavior; NC license blocks commercial training. |
| Aria Everyday Activities (AEA) | ~7.3 h · Project Aria glasses | RGB + IMU + eye + SLAM | Non-commercial research | No | Tiny (7.3 h); Aria-hardware-specific optics; NC — cannot ship inside a product. |
| Nymeria | Large-scale · Aria glasses + full-body mocap, in the wild | RGB + IMU + full-body motion capture | Non-commercial research | No | Motion-capture focus; sparse object and appliance-state labels; NC. |
| EgoLife | Multi-day · multi-person life-logging | Wearable RGB + audio | MIT on card (verify) | Conditional | Card claims MIT, but the capture carries heavy bystander/PII exposure and an unverified consent posture — a legal risk for commercial use. |
Open datasets vs Truelabel custom capture
The license wall: every natural-behavior home corpus above is non-commercial (Charades-Ego, Aria AEA, Nymeria) or has an unverifiable consent posture (EgoLife, UCI ADL). There is effectively no annotated, commercially-licensed egocentric ADL corpus you can legally train a shipped product on — custom capture is the only clean path.
Acted vs lived-in: Charades-Ego is scripted from prompts and academic captures are short and staged. Home robots fail on the clutter, lighting, and improvised sequences of a real household; custom capture collects unscripted routines in real homes.
Taxonomy and appliance control: your action and object taxonomy — your appliance models, your cleaning-product SKUs, your target rooms — is fixed at spec time, not inherited from a 2012 label set of 18 actions.
Consent chain and exclusivity: every clip ships with a documented wearer and bystander consent artifact plus a PII-blur pass, and you can negotiate exclusivity so the same home footage is never resold to a competitor.
Home & activities of daily living (ADL): by the numbers
The figures below are specific to home & activities of daily living (ADL) egocentric data and anchor the comparisons above.
- UCI ADL 2012: ~1 million frames, 18 ADL actions, 20 subjects, chest-mounted
- Charades-Ego: 7,860 videos across 157 action classes with paired first/third-person views
- Aria Everyday Activities: ~7.3 hours of everyday-activity egocentric recordings
- Nymeria: Aria glasses paired with full-body motion capture, captured in the wild
- EgoLife: multi-day, multi-person life-logging capture with an unverified consent posture
How Truelabel captures home & activities of daily living (ADL) data
Truelabel runs home & activities of daily living (ADL) programs on a network of 20,000+ consented collectors across nine countries, capturing to your brief on a head-mounted rig. Every clip passes per-clip machine QA — head-mount stability, field of view, and hands-in-frame coverage — and ships with a signed wearer consent artifact and provenance manifest. A calibration pilot returns its first batch in days, then accepted batches scale to 40–200 accepted hours per program, delivered as H.265 clips + per-clip JSON (actions, objects, appliance states, room, device, consent id); Hugging Face-streamable. Go deeper via what egocentric data means for physical AI, egocentric data licensing and commercial rights, sourcing egocentric kitchen and cooking video, egocentric warehouse video sourcing, and industrial egocentric video sourcing.
Related pages
Use these to move from category-level context into specific task, dataset, format, and comparison detail.
External references and source context
- EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data
EgoScale supports the claim that dexterous-manipulation performance scales with the volume and diversity of egocentric human demonstrations.
arXiv ↩ - AoE: Always-on Egocentric Human Video Collection for Embodied AI
AoE frames scalable, low-cost collection of egocentric human video as the answer to the scarcity of real-world task data for humanoids.
arXiv ↩ - EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video
EgoDex is a large-scale egocentric video corpus used as a pretraining substrate for dexterous manipulation.
arXiv ↩ - Detecting Activities of Daily Living in First-Person Camera Views (ADL dataset)
The 2012 UCI ADL project page backs the chest-mounted, license-unclear, acted-behavior characterization of the original ADL corpus.
UC Irvine (Pirsiavash & Ramanan) - Charades-Ego: paired first- and third-person activity videos
The Charades-Ego project page backs the scripted, paired ego/exo, non-commercial characterization.
Allen Institute for AI (PRIOR) - Aria Everyday Activities (AEA)
The Aria Everyday Activities dataset page backs the ~7.3-hour size and non-commercial research license.
Meta / Project Aria - Nymeria: egocentric full-body motion dataset
The Nymeria dataset page backs the Aria-plus-full-body-mocap, in-the-wild, non-commercial characterization.
Meta / Project Aria - EgoLife
The EgoLife project page backs the multi-day multi-person life-logging capture and the MIT-card-with-privacy-exposure caveat.
EgoLife project
FAQ
What is an ADL dataset?
An ADL (activities of daily living) dataset is first-person video of routine household tasks — cooking, cleaning, grooming, eating, dressing — labeled with the action and the objects involved. Robotics teams use it to pretrain home-manipulation policies; health and ambient-assisted-living teams use it for activity monitoring and decline or fall detection.
Why not just use the UCI ADL dataset or Charades-Ego for free?
The 2012 UCI ADL corpus is chest-mounted SD video with no clear license, and Charades-Ego is scripted and non-commercial. Neither is head-mounted, natural, or legally clean for training a shipped product. They are fine for benchmarking terminology and tasks, not for commercial home-robot training.
Is there any commercially licensed egocentric ADL dataset?
Effectively no. The natural-behavior home corpora — Aria Everyday Activities and Nymeria — are non-commercial research licenses, and EgoLife's MIT card sits on top of heavy bystander and PII exposure. Commercial-grade ADL data with a documented consent chain today means custom capture.
How do you handle consent and privacy inside someone's home?
Every wearer signs a consent agreement, and bystanders (household members, visitors) are consented or blurred. Each clip carries a consent artifact plus a PII pass before delivery — faces, screens, documents, and mail are blurred by default.
Do we get exclusive rights, or will the footage be resold?
You can license custom home ADL capture exclusively, so the same footage is not sold to competitors. You also set the action and object taxonomy, target rooms, and demographic mix at spec time.
Do you capture IMU, gaze, or hand pose alongside RGB?
Yes. RGB at 1080p/30 is the baseline; head IMU, gaze, and stereo/depth for 3D hand-object pose (25 joints per hand) are add-ons. Labels are frame-aligned action segments plus object and appliance-state changes.
Looking for adl dataset?
Specify modality, task, environment, rights, and delivery format. Truelabel matches you with vetted capture partners and helps scope consent artifacts and commercial licensing requirements before delivery.
Request ADL egocentric data