Existing motion-language models often lose timing information by compressing motion clips or concatenating pose features without retaining when each pose occurs. This makes action duration and temporal scale ambiguous, especially under temporal subsampling. FiGMo addresses this by representing motion as timestamp-grounded skeletal poses, allowing the model to reason explicitly about action order and duration.

The model uses a unified pose encoder that works for both 2D and 3D skeletons, including poses detected from real videos, with RGB optionally providing contextual information. Because existing datasets provide limited pose-level grounding, the authors constructed their own pose-to-motion supervision to train the model to connect individual body configurations to their temporal context.

Controlled experiments show that timestamps improve temporal localization and preserve physical timing under subsampling, while the constructed supervision is the main driver of performance gains. FiGMo achieves state-of-the-art results across motion question answering, dense motion captioning, and action recognition with detected poses. The paper has been accepted at WACV 2027.