DATASET FACET · TASK
Human-object interaction datasets for physical AI
Human hands, bodies, tools, and objects interacting in real scenes.
DIRECT ANSWER
Human-object interaction pages collect datasets where this taskis materially relevant, then add truelabel’s commercial use, consent risk, and deployment fit notes so buyers can decide whether public data is enough.
MATCHED DATASETS
7 catalog entries
7 of 7 datasets
Ego4D
Published May 2026 · custom
A large-scale egocentric video dataset focused on first-person human activity understanding.
- Large-scale first-person video corpus with annotations; source controls exact access terms.
- Commercial use restricted
- Best for: egocentric perception
EPIC-KITCHENS
Published May 2026 · custom
An egocentric video dataset of kitchen activities used for action recognition and human-object interaction research.
- Large egocentric kitchen activity dataset with action annotations.
- Commercial use restricted
- Best for: egocentric action recognition
HOI4D
Published May 2026 · custom
A 4D egocentric human-object interaction dataset with RGB-D and pose-oriented annotations.
- Human-object interaction dataset with 4D annotations; exact release details live on the source site.
- Commercial use unclear
- Best for: hand-object perception
DexYCB
Published May 2026 · custom
A dexterous hand-object interaction dataset centered on grasping YCB objects with 3D annotations.
- Dexterous grasping dataset with YCB objects and pose labels.
- Commercial use unclear
- Best for: hand pose estimation
Something-Something V2
Published May 2026 · custom
A human action video dataset focused on object interactions and temporal reasoning.
- Large human-object action video dataset; access and current terms are controlled by the dataset host.
- Commercial use restricted
- Best for: temporal action recognition
DexMV
Published May 2026 · custom
A dexterous manipulation dataset focused on multi-view visual observations and hand-object interaction.
- Dexterous manipulation demonstrations and visual observations; verify source for release details.
- Commercial use unclear
- Best for: dexterous manipulation
Kinetics
Published May 2026 · custom
A large video action recognition dataset used widely for video model pretraining.
- Large action recognition video corpus maintained through public benchmark releases.
- Commercial use restricted
- Best for: video pretraining
READ THE TAG WITH CARE
Do not treat this tag as the whole sourcing decision
Facet groupings are discovery aids, not final recommendations. A shared modality, task, robot, format, license, or commercial-use label only says that datasets are worth comparing; it does not prove that the source is safe, complete, or useful for a target model.
Use this grouping to shortlist candidates, then open the dataset profiles, run fit and license checks, and compare sources against the buyer's target environment. Thin tag results become useful only when they route the reader into deeper evidence and action surfaces.
The external references below keep the facet grounded in robotics data practice. They help reviewers understand why format, embodiment, trajectory quality, licensing, and real-world coverage matter before a team commits engineering time to ingestion.
When a facet has only a few matching datasets, treat that as a signal rather than a weakness. It may mean the public corpus is thin for that robot, task, or format, and the next move is a custom supplement with the facet written into acceptance criteria.
Where to go next
- Physical AI dataset catalogUse the catalog to compare source-backed dataset profiles by modality, task, rights signal, consent risk, and deployment fit.
- Hugging Face robotics indexScan the broader robotics dataset surface before narrowing into promoted profiles, comparisons, and custom collection specs.
- Dataset changelogTrack source updates, licensing notes, and buyer-readiness changes that should trigger a renewed review.
- Dataset fit checkerScore whether a public source is enough for the model, rights path, modalities, and target environment.
- License risk checkerSeparate source license language from contributor consent, redistribution, private-space risk, and model-use assumptions.
- Data spec generatorTurn a public-source gap into a scoped capture request with sample QA, metadata, and delivery requirements.
- Vendor alternatives hubCompare data providers when the answer is not another public dataset but a better sourcing or capture route.
- Data annotation companiesUse the company index to separate annotation vendors, data engines, marketplaces, and specialist capture teams.
Other places to verify the claims
- Scale AI physical AI data engineMarket context for why physical AI systems need custom, enriched, real-world data beyond generic labeling workflows.
- LeRobot documentationRobotics dataset and tooling context for Hugging Face based collection, sharing, conversion, and training workflows.
- Open X-EmbodimentA cross-embodiment robotics dataset reference for comparing trajectory scale, robot diversity, and VLA training assumptions.
- DROID datasetA large in-the-wild robot manipulation dataset reference for real-world trajectory capture and deployment transfer risk.
TRUELABEL ROUTING
Need human-object interaction data with cleaner rights?
Commission target-scoped data with contributor consent, capture metadata, sample QA, and delivery into your own storage.