Industry · Egocentric data
Egocentric Video Data for Agriculture Robotics
An agriculture robotics dataset for egocentric AI is first-person, head-mounted video of farm work — picking, pruning, sorting and inspecting crops from the worker's own point of view. As of mid-2026 no commercially-usable egocentric agriculture corpus exists: the nearest open data is either ground-vehicle-mounted (Rosario v2), factory-floor manual work (Egocentric-10K), or an incidental outdoor slice of a signed-license corpus (Ego4D). That leaves consented, seasonal custom capture across a multi-country collector network as the only route to human-POV farm training data.
Quick facts
- Resolution
- 1080p @ 30fps baseline; 2160p for fine ripeness- and defect-judgement steps
- Field of view
- ≥120° horizontal — keeps both hands and the plant/produce in frame at arm's length
- Mount
- Head-mounted hat or cap rig, outdoor-rated for sun, dust and rain; never chest-mount — the camera must follow gaze to the fruit or cut point
- Sensors
- RGB (baseline), IMU — head, optional wrist for reach and cutting motion, GPS / row-and-block context (optional, for field and cultivar tagging), Depth (optional, iPhone-Pro-class LiDAR for canopy and reach geometry)
- Labels
- Frame-aligned task/step segments (reach, grasp, pick, cut, place, sort); Crop and produce states in frame (ripe / unripe / defective / diseased); Tool-in-use spans (shears, knife, bag, tote)
- Volume
- 40–200 accepted hours per crop/task program, scheduled to the harvest or pruning window; calibration pilot batch in days
Key papers
Hard citations for the claims above. Each entry pairs a specific number with the paper that reports it.
The Rosario Dataset v2: Multimodal Dataset for Agricultural Robotics
2+ hours, ground-vehicle. The Rosario Dataset v2 is a multimodal agricultural-robotics dataset — over two hours from a soybean field with stereo, IMU, GNSS and wheel odometry and 6-DOF ground truth — captured from a ground vehicle rather than a human egocentric viewpoint, the nearest open adjacency to first-person farm-work data.
EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video
829 hours, 194 tasks. EgoDex pairs 829 hours of egocentric video across 194 tabletop tasks with 3D hand and finger tracking captured on Apple Vision Pro — the largest and most diverse dexterous human-manipulation dataset to date.
Ego4D: Around the World in 3,000 Hours of Egocentric Video
3,670 hours, 74 locations. Ego4D spans 3,670 hours of daily-life first-person video from 931 camera wearers across 74 locations in 9 countries, collected under consenting-participant privacy and de-identification standards.
What agriculture robotics needs from egocentric data
An agriculture robotics dataset for egocentric AI is first-person, head-mounted video of farm work — picking, pruning, sorting and inspecting crops from the worker's own point of view. As of mid-2026 no commercially-usable egocentric agriculture corpus exists: the nearest open data is either ground-vehicle-mounted (Rosario v2), factory-floor manual work (Egocentric-10K), or an incidental outdoor slice of a signed-license corpus (Ego4D). That leaves consented, seasonal custom capture across a multi-country collector network as the only route to human-POV farm training data.
The capture settings this covers:
- Harvest picking: selecting and detaching ripe produce — berries, tomatoes, grapes, tree fruit — by hand, judging ripeness from the picker's own viewpoint
- Pruning and thinning: cutting canes, suckers and excess fruit with shears, including two-handed branch and vine manipulation at close range
- Post-harvest sorting and grading: hand-culling on a packing line by size, colour and defect, with produce turned and inspected in the palm
- Crop and plant scouting: walking rows to spot pests, disease and ripeness, turning leaves and examining stems and undersides
- Gloved manipulation and tool use in frame: pruning shears, harvest knives, twine, vine clips and picking bags grasped and used under gloves and dirt
- Field material handling: filling, carrying and emptying harvest totes, crates and bins between rows and transport points
Why agriculture robotics needs first-person human video
Apple's EgoDex pretrains dexterous manipulation on large-scale egocentric human video, so a farm worker's first-person view of picking and pruning sits in the same training substrate an agriculture-robot policy learns from. [1]
EgoScale shows dexterous-manipulation performance scaling with both the volume and the diversity of egocentric human data, which is exactly what a crop- and region-spanning farm corpus is built to supply. [2]
AoE frames low-cost, scene-agnostic collection of egocentric human video of manual tasks as the practical answer to embodied-AI data scarcity — the whitespace an agriculture corpus with zero open equivalents has to close. [3]
Capture and delivery spec
Every agriculture robotics capture program runs to an explicit spec so the footage is training-ready on delivery rather than after a re-shoot. The baseline below is tuned per program; sensors, labels, and volume scale with the buyer's model.
| Spec | Detail |
|---|---|
| Resolution | 1080p @ 30fps baseline; 2160p for fine ripeness- and defect-judgement steps |
| Frame rate | 30fps baseline, 60fps for fast picking and cutting motions |
| Field of view | ≥120° horizontal — keeps both hands and the plant/produce in frame at arm's length |
| Mount | Head-mounted hat or cap rig, outdoor-rated for sun, dust and rain; never chest-mount — the camera must follow gaze to the fruit or cut point |
| Sensors | RGB (baseline), IMU — head, optional wrist for reach and cutting motion, GPS / row-and-block context (optional, for field and cultivar tagging), Depth (optional, iPhone-Pro-class LiDAR for canopy and reach geometry) |
| Labels | Frame-aligned task/step segments (reach, grasp, pick, cut, place, sort); Crop and produce states in frame (ripe / unripe / defective / diseased); Tool-in-use spans (shears, knife, bag, tote); Crop, cultivar and growth-stage tags per clip; Optional 21-keypoint 3D hand pose per hand for gloved manipulation |
| QA gates | Hands-in-frame above threshold under gloves, dirt and occluding foliage; Outdoor-light exposure check — no blowout in direct sun, no crush in shade; Head-mount stability / no motion blur on the action frames; Crop and task match the brief (right crop, right growth stage, right operation); Per-clip wearer consent + grower/site release artifact attached |
| Delivery | H.265 + per-clip JSON metadata (steps, crop states, tools, hand pose, field tags), Hugging Face-streamable; consent and site-release artifacts shipped per clip |
| Volume | 40–200 accepted hours per crop/task program, scheduled to the harvest or pruning window; calibration pilot batch in days |
Open agriculture robotics datasets
The 3 open corpora most relevant to agriculture robotics are compared below on scale, sensors, license, commercial use, and the gap each leaves for a buyer. Only 1 of the 3 is permissively licensed for commercial use — which is the whole reason custom capture exists.
| Dataset | Size / scale | Sensors | License | Commercial use | Gap |
|---|---|---|---|---|---|
| Rosario Dataset v2 | Multimodal agricultural-robotics field data (stereo, GNSS, IMU, wheel odometry) | Ground-vehicle-mounted rig — NOT head-worn / egocentric | Research (academic); commercial terms require negotiation | Conditional | It is the robot's-eye view from a field platform, not the worker's point of view — no hands-in-frame manipulation of crops, and no consent chain built for commercial training. |
| Ego4D (outdoor / farm-adjacent slices) | Signed-license egocentric corpus; only a thin, incidental outdoor slice touches farm work | Head-mounted RGB (audio / IMU on some subsets) | Ego4D signed license | Conditional | Not an agriculture dataset — outdoor clips are incidental, carry no farm-task labels, and the signed license restricts redistribution and commercial reuse. |
| Egocentric-10K | 10,000 h · 2,138 workers · 87 factories | Head-mounted RGB (~1080p); no depth/IMU/gaze, no labels | Apache 2.0 | Yes | The nearest permissively-licensed worn-camera adjacency, but it is factory-floor manual work — no crops, no outdoor light, no farm tasks, and no labels to learn from. |
Open datasets vs Truelabel custom capture
Whitespace, not a discount: there is no commercially-usable egocentric agriculture corpus to buy or benchmark against. Rosario v2 is vehicle-mounted, Ego4D's outdoor clips are incidental and signed-license, and Egocentric-10K is factory manual work. Custom capture is not the cheaper option here — it is the only way to get human-POV farm footage at all.
Seasonality and geography are the product, not a constraint: crops ripen in narrow windows and vary by region and cultivar. A LATAM and multi-country collector network can put the right crop at the right growth stage in front of a camera on demand — coverage that any fixed, one-off academic collection can never match.
Outdoor manipulation QA the open sets never had to solve: gloves, dirt, occluding foliage and direct sun wreck hands-in-frame coverage and exposure. Field-tuned per-clip QA gates are what make farm footage trainable rather than just recorded.
Exclusivity and a real consent chain: custom farm capture ships a signed wearer consent and grower/site release on every clip and is exclusive to your program — neither of which a scraped or academic outdoor corpus can offer.
Agriculture robotics: by the numbers
The figures below are specific to agriculture robotics egocentric data and anchor the comparisons above.
- Zero commercially-usable egocentric agriculture corpora exist as of July 2026 — the entire category is whitespace (verified by absence at build)
- Physical Intelligence: adding egocentric human video delivered roughly a 2x improvement on tasks where robot data was scarce (Dec 2025)
- Rosario Dataset v2 is multimodal but ground-vehicle-mounted, not human-POV (arXiv:2508.21635)
- Egocentric-10K: 10,000 h of Apache-2.0 worn-camera manual work — the nearest permissive adjacency, with zero farm scenes
- "precision agriculture dataset" draws about 10 searches/mo (DataForSEO, 2026-07-06), and the egocentric-farm phrasing returns an all-academic SERP
- TrueLabel agriculture programs: 40–200 accepted hours per crop/task, scheduled to the harvest or pruning window
How Truelabel captures agriculture robotics data
Truelabel runs agriculture robotics programs on a network of 20,000+ consented collectors across nine countries, capturing to your brief on a head-mounted rig. Every clip passes per-clip machine QA — head-mount stability, field of view, and hands-in-frame coverage — and ships with a signed wearer consent artifact and provenance manifest. A calibration pilot returns its first batch in days, then accepted batches scale to 40–200 accepted hours per crop/task program, scheduled to the harvest or pruning window; calibration pilot batch in days, delivered as H.265 + per-clip JSON metadata (steps, crop states, tools, hand pose, field tags), Hugging Face-streamable; consent and site-release artifacts shipped per clip. Go deeper via what egocentric data is, egocentric data licensing, VLA training data, cross-embodiment transfer, custom data sourcing, the physical AI data marketplace, and humanoid robot training data.
Related pages
Use these to move from category-level context into specific task, dataset, format, and comparison detail.
External references and source context
- EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video
EgoDex is a large-scale egocentric video corpus built to pretrain dexterous manipulation from first-person human video.
arXiv ↩ - EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data
EgoScale shows dexterous-manipulation performance scaling with the volume and diversity of egocentric human data.
arXiv ↩ - AoE: Always-on Egocentric Human Video Collection for Embodied AI
AoE frames scalable, low-cost collection of egocentric human video of manual tasks as an answer to embodied-AI data scarcity.
arXiv ↩ - The Rosario Dataset v2: Multimodal Dataset for Agricultural Robotics
Rosario Dataset v2 is a multimodal agricultural-robotics dataset captured from a ground vehicle, not from a human egocentric viewpoint.
arXiv - Egocentric video remains useful but incomplete for robot data buyers
Ego4D is a large signed-license egocentric corpus whose outdoor footage is incidental and not labelled for agricultural tasks.
ego4d-data.org - Egocentric-10K dataset card and license
The Egocentric-10K dataset card is the source for the Apache-2.0 license and the 10,000 h / 2,138 workers / 87 factories figures.
Hugging Face
FAQ
There is no open egocentric farm dataset — what do we actually start from?
You start from a spec, not a download. Because no commercially-usable egocentric agriculture corpus exists, we scope the crops, tasks and growth stages you need, run a small calibration pilot, then scale accepted batches. The nearest open data (Rosario v2, Ego4D outdoor slices, Egocentric-10K) is useful context but none of it is human-POV farm work you can train and ship on.
Can you capture seasonal and regional crop diversity on demand?
Yes — that is the core advantage of a collector network over a fixed dataset. Crops ripen in windows and differ by region and cultivar, so we schedule capture to the harvest or pruning calendar and draw on a LATAM and multi-country network to reach the crop and growth stage your model needs, tagged per clip.
Which farm tasks can you capture — picking, pruning, sorting, inspection?
All four, plus gloved tool use and field material handling. Every clip carries frame-aligned task/step labels (reach, grasp, pick, cut, place, sort), crop-state labels (ripe, unripe, defective, diseased) and tool-in-use spans, so the footage matches the operation your robot has to perform.
How do gloves, dirt and harsh outdoor light affect your hands-in-frame QA gates?
They are exactly what the field QA gates are tuned for. We enforce a hands-in-frame threshold that holds under gloves, dirt and occluding foliage, plus an exposure check that rejects blown-out sun and crushed shadow, and a stability gate against motion blur — the checks the indoor open corpora never had to solve.
Why does agriculture robotics need human-POV data when field robots are vehicle-mounted?
Because the manipulation happens at the hand, not the chassis. Selective harvesting, pruning and grading are dexterous, close-range tasks; a ground-vehicle view like Rosario v2 sees the field but not the fingertip decisions. First-person human video captures the reach, grasp and ripeness judgement a manipulation policy has to imitate.
Do we get exclusive rights to footage of our crop and our tasks?
Yes. Custom-capture programs are exclusive by default: the footage is yours, dated to your crop and season, and not resold — unlike a public dataset every competitor can already ingest. Each clip also ships with a wearer-consent and grower/site release, an auditable chain the open corpora lack.
Looking for agriculture robotics dataset?
Specify modality, task, environment, rights, and delivery format. Truelabel matches you with vetted capture partners and helps scope consent artifacts and commercial licensing requirements before delivery.
Request agriculture egocentric data