Robots working alongside people need to handle two conflicting demands: they must compensate for unknown payloads, such as a box they are lifting, while remaining compliant when a human pushes or guides them. The authors of this paper argue that no single sensor can reliably distinguish these cases. Force readings cannot separate a payload from a human push, and vision alone is vulnerable to occlusions.
To address this, the team proposes a task-agnostic framework built around a transformer-based classifier that fuses force history and 3D point-cloud data. The classifier is trained with modality dropout, so it learns to work even when one input stream is missing, and it is fine-tuned from simulation to real hardware. In experiments it reached 91.89% accuracy and stayed above 90% when one modality was completely occluded.
The classifier drives a system-identification module that selectively compensates for payloads and a variable-impedance controller that switches between adaptive and reactive modes. It also estimates interaction forces for downstream tasks such as human intent estimation. The authors validated the system under external perturbations and unknown loads, including single-arm manipulation, an imitation-learning-based policy, intent-driven target switching, and zero-shot bimanual deployment on two Franka arms lifting a box of unknown mass.