Real-time monocular human pose estimation is a key component of human-centered intelligent systems, but camera-only approaches struggle when visual evidence is incomplete. The authors of a new preprint note that lower-body occlusion or brief moments when the subject leaves the frame often lead to physically implausible leg motion and unreliable foot-ground contact.
To address this, the researchers collected a synchronized multimodal dataset and built a part-decoupled temporal fusion framework. The model fuses foot-sensing features from smart insoles—measuring plantar pressure and acceleration—into the lower-body branch, while a lightweight coordinator exchanges information between upper- and lower-body temporal representations to preserve whole-body consistency.
The framework was evaluated with three representative real-time visual backbones based on 2D keypoints, 3D landmarks, and SMPL-based mesh recovery. Across all three, incorporating the simple foot sensors improved pose accuracy, motion smoothness, and gait-related measurements compared with vision-only baselines, with the largest gains appearing under occlusion and short visual-dropout conditions.
The results suggest that unobtrusive foot sensing can provide complementary physical observability exactly when camera observations are temporarily unavailable, pointing toward more robust pose estimation in real-world settings.