Environment · Egocentric data
Egocentric Video Data for Retail & Store Operations
Retail egocentric video data is head-mounted, first-person footage of store work — restocking shelves, checkout and scanning, backroom receiving, and customer-assisted shopping. No commercially-licensed open egocentric retail corpus exists today, so consented custom capture is the only compliant path to store-operations training data.
Quick facts
- Resolution
- 1080p @ 30fps baseline; stereo 2160p @ 60fps available for shelf-depth and reach-distance studies
- Field of view
- ≥120° horizontal so both the shelf/display face and the associate's hands stay in frame during reach-and-grasp
- Mount
- Head-mounted (cap clip or glasses rig) — never chest or handheld, which lose overhead-shelf context and hand detail at the fixture
- Sensors
- RGB (primary), IMU for head motion and aisle-to-aisle gait, Optional gaze for attention-to-shelf and search behavior, Optional depth for reach distance and shelf-facing geometry
- Labels
- Frame-aligned action segments (reach, grasp, scan, bag, restock, face, return-to-shelf); SKU and product-category tags per interaction; Planogram-compliance and facing states
- Volume
- 40–100 accepted hours per pilot across 8–15 store formats
Key papers
Hard citations for the claims above. Each entry pairs a specific number with the paper that reports it.
EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video
829 hours, 194 tasks. EgoDex pairs 829 hours of egocentric video across 194 tabletop tasks with 3D hand and finger tracking captured on Apple Vision Pro — the largest and most diverse dexterous human-manipulation dataset to date.
Ego4D: Around the World in 3,000 Hours of Egocentric Video
3,670 hours, 74 locations. Ego4D spans 3,670 hours of daily-life first-person video from 931 camera wearers across 74 locations in 9 countries, collected under consenting-participant privacy and de-identification standards.
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
1,286 hours, 740 participants. Ego-Exo4D pairs simultaneously-captured egocentric and exocentric video of skilled activity from 740 participants across 13 cities — 1,286 hours with multichannel audio, eye gaze, 3D point clouds, camera poses, and IMU.
What retail & store operations egocentric data captures
Retail egocentric video data is head-mounted, first-person footage of store work — restocking shelves, checkout and scanning, backroom receiving, and customer-assisted shopping. No commercially-licensed open egocentric retail corpus exists today, so consented custom capture is the only compliant path to store-operations training data.
The capture settings this covers:
- Shelf restocking and facing — an associate pulls stock from a rolling cart, faces product, rotates dated items, and rebuilds a planogram after a busy weekend.
- Checkout and point-of-sale — scanning items across the belt, handling barcodes and produce lookups, bagging, and operating the payment terminal.
- Backroom receiving and inventory — unloading pallets, scanning inbound cases, cycle counts, and moving totes between the stockroom and the sales floor.
- Customer-assisted shopping — locating a SKU on request, comparing two products, reading nutrition or spec labels, and loading a cart or basket.
- Returns and go-backs — inspecting returned items, re-tagging, and walking product back to its home location across departments.
- Fresh, deli, and food-service counters — weighing, wrapping, slicing, and label printing in the perishable sections where hand-object interaction is densest.
Why robotics and AI labs need retail & store operations data
Retail robots that stock shelves, scan planograms, and fetch items for shoppers learn fastest from first-person human demonstrations, and Apple's EgoDex shows that large egocentric human corpora transfer directly into robot manipulation policies. [1]
EgoLive-scale corpora of real-world human tasks are the same first-person modality a store-associate head camera captures on the sales floor — the base layer of humanoid pretraining. [2]
The binding constraint on physical AI is real-world interaction data, not model capacity — the AoE line of work frames scalable egocentric capture as the answer — and store operations are one of the largest commercial domains with essentially zero captured first-person footage. [3]
EgoScale reports that manipulation performance scales with the diversity of egocentric human data, and retail spans thousands of SKUs, packagings, and store formats that no fixed academic corpus covers. [4]
Capture and delivery spec
Every retail & store operations capture program runs to an explicit spec so the footage is training-ready on delivery rather than after a re-shoot. The baseline below is tuned per program; sensors, labels, and volume scale with the buyer's model.
| Spec | Detail |
|---|---|
| Resolution | 1080p @ 30fps baseline; stereo 2160p @ 60fps available for shelf-depth and reach-distance studies |
| Frame rate | 30fps baseline; 60fps for fast checkout scanning and bagging motions |
| Field of view | ≥120° horizontal so both the shelf/display face and the associate's hands stay in frame during reach-and-grasp |
| Mount | Head-mounted (cap clip or glasses rig) — never chest or handheld, which lose overhead-shelf context and hand detail at the fixture |
| Sensors | RGB (primary), IMU for head motion and aisle-to-aisle gait, Optional gaze for attention-to-shelf and search behavior, Optional depth for reach distance and shelf-facing geometry |
| Labels | Frame-aligned action segments (reach, grasp, scan, bag, restock, face, return-to-shelf); SKU and product-category tags per interaction; Planogram-compliance and facing states; Optional 2D/3D hand pose on interaction segments; Bystander and PII redaction masks (faces, payment screens, loyalty cards) |
| QA gates | Hands-in-frame on every interaction segment; SKU / label legible with no motion blur on the product; Head stability within sway threshold across aisle walking; Face and payment-screen redaction verified per clip; Signed consent artifact attached to every clip |
| Delivery | H.265 clips + per-clip JSON metadata (store format, region, banner, SKU set, action segments), Hugging-Face-streamable, with a consent and redaction manifest per clip |
| Volume | 40–100 accepted hours per pilot across 8–15 store formats |
Open retail & store operations datasets
The 5 open corpora most relevant to retail & store operations are compared below on scale, sensors, license, commercial use, and the gap each leaves for a buyer. Only 1 of the 5 is permissively licensed for commercial use — which is the whole reason custom capture exists.
| Dataset | Size / scale | Sensors | License | Commercial use | Gap |
|---|---|---|---|---|---|
| Egocentric-10K | 10,000 h · 2,138 workers · 87 factories | Head-mounted RGB, factory floor only | Apache 2.0 | Yes | The only large permissively-licensed egocentric corpus — and it contains zero retail-store footage; it is entirely manufacturing work with no SKU, planogram, or checkout content and no action labels. |
| EgoLive | Large-scale, real-world human tasks | Head-mounted RGB | Research; commercial terms unverified | Conditional | Claims broad real-world task coverage but publishes no retail-specific split and no verified commercial license or consent posture — treat any store-relevant slice as requiring negotiation with the authors. |
| Aria Everyday Activities (AEA) | ~7.3 h everyday indoor activities | Project Aria glasses (RGB, IMU, eye tracking) | Non-commercial research | No | Home and indoor everyday activities, not store operations; non-commercial license bars any production use, and there is no shopping, stocking, or checkout content. |
| Charades-Ego | Paired ego/exo scripted daily activities | Handheld / worn RGB | Non-commercial (default) | No | Scripted household activities filmed by crowd workers — no retail environment, no SKU taxonomy, and a non-commercial license. |
| EgoExoLearn | Paired ego/exo procedural demonstrations | Head-mounted RGB | Research-only | No | Lab and procedural skill-following tasks, not commercial store work; research-only terms and no consent chain for filming customers or staff. |
Open datasets vs Truelabel custom capture
There is no starting point in the open. Every other environment cell can at least compare against a nearest corpus; retail cannot, because no open egocentric store dataset exists and the one permissively-licensed corpus (Egocentric-10K) is 100% factory floor. Custom capture is not the better option — it is the only source of first-person store-floor data.
Retail footage is dense with people and PII: customers in aisles, staff badges, payment screens, and loyalty cards all enter frame. A defensible program needs a documented, per-clip consent chain plus verified face and payment-screen redaction — artifacts no academic dataset provides and no scrape can produce lawfully.
Buyers can specify the store: banner, format (grocery, apparel, convenience, pharmacy, warehouse club), region, and the exact SKU set and planograms to exercise. A fixed corpus freezes one setting; custom capture matches your deployment environment and object catalog.
Planograms, packaging, and seasonal assortments change constantly, so a 2012-era or one-off academic snapshot is stale on arrival. Custom capture delivers current footage and, when required, exclusive rights so the same store-floor demonstrations are never resold to a competitor.
Retail & store operations: by the numbers
The figures below are specific to retail & store operations egocentric data and anchor the comparisons above.
- 0 commercially-licensed open egocentric retail datasets exist as of 2026 — retail is the family's clearest whitespace cell.
- Egocentric-10K's 10,000 permissively-licensed hours contain 0 retail-store hours; it is 100% factory floor across 2,138 workers and 87 factories.
- A retail pilot batch = 40–100 accepted store-floor hours captured across 8–15 store formats.
- 3+ SKU-interaction states are labeled per shopper/associate action (reach, grasp, scan, restock, return-to-shelf), aligned frame-by-frame.
How Truelabel captures retail & store operations data
Truelabel runs retail & store operations programs on a network of 20,000+ consented collectors across nine countries, capturing to your brief on a head-mounted rig. Every clip passes per-clip machine QA — head-mount stability, field of view, and hands-in-frame coverage — and ships with a signed wearer consent artifact and provenance manifest. A calibration pilot returns its first batch in days, then accepted batches scale to 40–100 accepted hours per pilot across 8–15 store formats, delivered as H.265 clips + per-clip JSON metadata (store format, region, banner, SKU set, action segments), Hugging-Face-streamable, with a consent and redaction manifest per clip. Go deeper via egocentric data licensing, what egocentric data is, industrial egocentric video sourcing, warehouse egocentric video, and kitchen egocentric video.
Related pages
Use these to move from category-level context into specific task, dataset, format, and comparison detail.
External references and source context
- EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video
EgoDex is a large-scale egocentric human video corpus that transfers into robot manipulation policies.
arXiv ↩ - EgoLive: A Large-Scale Egocentric Dataset from Real-World Human Tasks
EgoLive is a large-scale egocentric dataset of real-world human tasks, the same first-person modality a store-associate camera captures.
arXiv ↩ - AoE: Always-on Egocentric Human Video Collection for Embodied AI
AoE frames scalable, low-cost collection of egocentric human video as the answer to the shortage of real-world interaction data.
arXiv ↩ - EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data
EgoScale supports the claim that manipulation performance scales with the diversity of egocentric human data, motivating retail's broad SKU and store-format coverage.
arXiv ↩ - Egocentric-10K
Egocentric-10K is the source for the 10,000-hour / 2,138-worker / 87-factory scale and its factory-only, retail-absent scope.
Hugging Face - Egocentric-10K dataset card and license
The Egocentric-10K dataset card and license anchor the Apache-2.0 commercial-use characterization used in the open-datasets table.
Hugging Face - EgoLive: A Large-Scale Egocentric Dataset from Real-World Human Tasks
EgoLive is the source for the nearest real-world-task corpus whose retail relevance and commercial terms are unverified.
arXiv - Aria Everyday Activities (AEA)
Aria Everyday Activities supports the ~7.3-hour everyday-indoor, non-commercial characterization in the open-datasets table.
Meta / Project Aria - Charades-Ego: paired first- and third-person activity videos
Charades-Ego supports the scripted-household, non-commercial nearest-adjacent row in the open-datasets table.
Allen Institute for AI (PRIOR) - EgoExoLearn: bridging egocentric and exocentric skill learning
EgoExoLearn supports the paired ego/exo procedural-demonstration, research-only nearest-adjacent row in the open-datasets table.
OpenGVLab
FAQ
Is there an open-source egocentric retail dataset we can just use?
No. As of 2026 there is no commercially-licensed open egocentric corpus of store operations. The largest permissively-licensed first-person corpus, Egocentric-10K, is entirely factory-floor work and contains no retail footage. Everything else that touches store-relevant tasks (Aria Everyday Activities, Charades-Ego, EgoExoLearn) is non-commercial or research-only and still not filmed in a store. EgoLive claims broad real-world task coverage, but its retail relevance and commercial terms are unverified. Retail is genuine whitespace, which is why custom capture is the only compliant path.
How do you handle consent and PII when filming inside a store with customers and staff?
Every wearer signs a consent artifact that is attached to each clip, and capture happens under a store agreement that covers the environment and staff. Bystanders — customers in the background, faces, name badges, payment terminals, and loyalty cards — are redacted in a dedicated QA pass, and each clip ships with a redaction manifest. This consent-and-provenance chain is exactly what no scraped or academic dataset can give a buyer, and it is the difference between a dataset your legal team can approve and one it cannot.
Can you match our store formats, banners, and SKU set?
Yes. Because capture is bespoke, you specify the store format (grocery, apparel, convenience, pharmacy, warehouse club), the banner, the region, and the SKU set or planograms you want exercised. A typical pilot is 40–100 accepted hours across 8–15 store formats, and each clip carries store-format, region, banner, and SKU-set metadata so you can filter and re-balance the mix. A frozen public corpus cannot be steered this way.
Do we get exclusive rights, or will the footage be resold to competitors?
Both models are available. Standard programs license the footage to you with clear commercial rights; exclusive programs guarantee the store-floor demonstrations you commissioned are never sold to another buyer. Because retail assortments and planograms change seasonally, exclusivity plus fresh capture also protects you from training on a stale snapshot that a rival could obtain later.
Why not just use factory (Egocentric-10K) or home (Aria) footage as a proxy?
The domain gap is too large to train reliable store behavior. Factory footage from Egocentric-10K shows fixed workstations and industrial parts, not shopper reaches, checkout scanning, or shelf facing; home footage from Aria Everyday Activities shows kitchens and living rooms under a non-commercial license. Retail-specific dynamics — dense SKUs, planogram compliance, cart and basket handling, and customer interaction — are absent from both, so a model pretrained on them still needs in-domain store data to perform.
Do you capture IMU, gaze, or hand pose alongside the RGB video?
Yes, on request. The RGB stream is primary at 1080p/30 (stereo 2160p/60 for reach-distance work). IMU is captured for head motion and aisle gait, optional gaze supports attention-to-shelf and product-search studies, and optional depth or 2D/3D hand pose can be added on interaction segments for grasp and reach modeling. Labels are frame-aligned action segments (reach, grasp, scan, bag, restock, face, return-to-shelf) with SKU and planogram-state tags.
Looking for retail egocentric video dataset?
Specify modality, task, environment, rights, and delivery format. Truelabel matches you with vetted capture partners and helps scope consent artifacts and commercial licensing requirements before delivery.
Request retail egocentric data