Tactile sensing can improve robot performance in contact-rich manipulation, but collecting real tactile feedback normally requires demonstrations captured with specialized tactile hardware. The FELT framework aims to sidestep that constraint by learning contact-aware representations from ordinary RGB observations, using existing paired visuo-tactile data as supervision.
FELT combines a frozen visual encoder with separate query decoders for the left and right finger, mirroring the spatial layout of dual-finger tactile arrays. Gated connections let the two branches exchange information, and a convolutional readout produces spatial contact representations. Downstream policies can consume these representations through two interfaces: one decodes them into tactile images, the other feeds the contact-aware features directly into the policy.
In experiments across four contact-rich real-robot tasks, both interfaces achieved higher success rates than vision-only policies. Directly incorporating the learned contact-aware features improved success by 15–22.5 percentage points, suggesting that useful tactile information can be synthesized from vision alone and reducing the need for tactile-equipped hardware at deployment.