Progress estimators provide dense reward signals for reinforcement learning in robotic manipulation, but they typically learn from expert demonstrations and assume every action contributes to success. That assumption often fails in real-world high-precision tasks, where subtle geometric errors or human mistakes can lead to failure. As a result, existing methods may misinterpret mistakes as meaningful progress and corrupt the learning signal.
The authors propose PRIME, a progress estimator that aggregates experience across multiple trajectories. It constructs a weighted graph over observed states, connecting states across trajectories based on spatial proximity and visual consistency. Edge weights encode spatial distances as transition costs, and a shortest-path search approximates the geodesic distance to success. Changes in this distance naturally distinguish advancement, stagnation, and setbacks, without assuming progress is monotonic over time.
In both simulation and real-world precise manipulation tasks, PRIME proved consistently effective and robust to varying degrees of suboptimality in demonstrations. Its advantage became more pronounced in higher-precision scenarios, yielding up to a 5x improvement in policy success rate over supervised fine-tuning baselines on charger insertion.