Contact-rich manipulation often requires knowing where and how hard a robot touches an object, but vision alone gives ambiguous information and tactile sensors are scarce and platform-specific. Ego2Contact instead learns to infer touch from ordinary RGB video, using a predictor trained on human egocentric recordings that were paired with tactile-glove measurements. The model fuses temporal scene context with hand-local visual features to estimate bimanual tactile fields.
The key step is transferring this human-learned prior to robot hands, which differ in appearance and tactile layout. Ego2Contact maps anatomical regions across species to align robot tactile layouts with the predictor's canonical grid, so robot measurements can fine-tune the model and its predictions can feed robot policies. The authors report that Ego2Contact outperforms earlier tactile predictors on human data, including unseen objects, and on robot data after adaptation.
In experiments, policies using predicted touch beat non-tactile baselines in simulation and recovered part of the benefit of ground-truth tactile sensing during real-robot post-training. The work suggests that scalable human video can supply a useful tactile signal when direct sensing is limited, though the authors note the adapted predictor does not fully replace physical tactile sensors.