Glossary
Sim-to-real gap
Sim-to-real gap means the performance gap between behavior learned in simulation and behavior deployed in real physical environments. The term matters because it turns a model or procurement concept into concrete data requirements you can evaluate samples against.
Quick facts
- Sim-only baseline
- RLBench — 100 hand-designed manipulation tasks in CoppeliaSim on a Franka Panda (RA-L 2020) — useful for ablations, useless for deployment risk.
- Real-world reference
- DROID — 76,000 trajectories / 350h / 564 scenes on the same Franka Panda embodiment as many sim envs (2024) — pairs cleanly with sim for sim-to-real diff measurement.
- Why sim alone fails
- Lighting, contact dynamics, sensor noise, and out-of-distribution objects diverge from the simulator; policies look strong in sim and fail at deployment.
- What to source
- Paired real-world episodes for the same tasks, with sim-to-real residual logged per episode so policy fitness is measured rather than assumed.
Comparison
| Question | Answer |
|---|---|
| Where it appears | Sourcing specs, QA requirements, dataset manifests, and buyer review notes |
| Why it matters | It turns abstract AI language into a supplier-verifiable requirement |
| Common failure | Using the term without defining modality, format, rights, or acceptance criteria |
How to use this term in a spec
The sim-to-real gap is the difference between model performance in simulation and performance after deployment in real physical conditions. Domain randomization work treats the gap as a transfer problem caused by visual and dynamics differences between simulated and real environments. [1]
What to avoid
Do not use sim-to-real gap as a vague keyword. Define the data files, metadata, rights, QA checks, and delivery format that make it measurable.
Sim-to-real gap in buyer review
Simulation benchmarks are useful for controlled experimentation, but buyers still need real complement data that stresses lighting, contact, clutter, and sensor artifacts. CALVIN, AI2-THOR, and ManiSkill illustrate how simulated environments support embodied AI research while still requiring careful transfer checks. [2] [3] [4]
Sim-to-real gap supplier evidence
Supplier evidence should include real-world samples that intentionally challenge the simulated assumptions. If the real sample only matches the easiest simulated scenes, it does not measure the transfer gap a buyer needs to close.
Related pages
Use these to move from category-level context into specific task, dataset, format, and comparison detail.
External references and source context
- Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
Domain randomization frames sim-to-real transfer as bridging differences between simulated and real-world visual environments.
arXiv ↩ - CALVIN paper
CALVIN is a simulated benchmark and toolkit for long-horizon language-conditioned robot manipulation.
arXiv ↩ - Project site
AI2-THOR provides simulated interactive household environments for embodied AI agents.
ai2thor.allenai.org ↩ - Project site
ManiSkill is a simulation benchmark for robot manipulation and embodied AI research.
maniskill.ai ↩
More glossary terms
FAQ
What is Sim-to-real gap?
Sim-to-real gap is the performance gap between behavior learned in simulation and behavior deployed in real physical environments.
Why does it matter for physical AI?
It matters because physical AI data must be connected to actions, environments, metadata, rights, and model use, not just raw files.
How should buyers spec it in a sourcing request?
Request real-world complement data that stresses lighting, contact, clutter, and object variation.
Can suppliers validate this from samples?
Yes, if the buyer defines visible evidence, metadata requirements, and acceptance criteria before suppliers submit files.
Find datasets covering sim to real gap
Truelabel surfaces vetted datasets and capture partners working with sim to real gap. Send the modality, scale, and rights you need and we route you to the closest match.
Request sim-to-real gap data