Robots that learn skills by watching humans could scale up data collection, but comparing such systems has been messy: each lab uses its own videos, robots, and environments. A new paper introduces RoboReel, a unified benchmark for learning-from-observation (LfO) that bundles real-world human demonstration videos, simulated robot trajectories, and evaluation environments across ten manipulation tasks.

The benchmark includes four test suites that measure different capabilities, including robustness to visual distractors and the ability to complete long-horizon tasks. The authors evaluate more than seven state-of-the-art algorithms, including vision-language-action (VLA) based variants, and study how different representation choices affect performance.

To check that simulation results transfer, the team recreated a subset of tasks in the real world and found that relative performance trends were preserved. Their analysis shows that long-horizon tasks and tasks with low tolerances remain difficult for current models, pointing to where the field needs to improve.