Recovering full-body 3D motion from sparse wearable sensors is difficult because real-world signals are noisy and often incomplete. Most existing models are tied to a fixed set of sensors, and diffusion-based approaches are too slow for real-time use. OmniPoser, a new framework from researchers Minghao Liu and Tutian Tang, instead accepts a flexible mix of inputs—such as 3D keypoints, sparse IMU rotations, or head-mounted 6-DoF poses—within a single model.

The key idea is a cross-modal masking mechanism paired with a Masked Conditioning Encoder. This lets the model handle whatever subset of sensors is available at inference time, without retraining. A decoupled dual-stream generative backbone, built on Conditional Flow Matching, produces complete SMPL motion in a single ODE integration step, avoiding the heavy iterative sampling of diffusion models.

Across three settings and seven benchmarks, including the in-the-wild Nymeria and Ego-Exo4D datasets, OmniPoser performs on par with state-of-the-art methods while running at over 700 FPS. The paper was accepted as a poster at ECCV 2026. The authors note that the approach is universal across heterogeneous inputs, though the source does not discuss limitations or failure cases.