Human mesh recovery (HMR) aims to reconstruct body and hand geometry from video, but occlusion and partial visibility in real scenes often degrade accuracy. Video-generation models learn rich spatial and temporal priors, yet those priors are tuned for visual synthesis rather than explicit human geometry. DreamHMR bridges that gap by first grounding video representations in human shape and then refining local details using reliable evidence from the scene.
The framework operates in two stages. Global Body Representation Transfer retrains the video backbone to predict complete human renderings, forcing it to learn surface correspondence and posed geometry. Local Joint Feature Enhancement then combines contextual estimates with trustworthy local cues, reducing interference from occluders and background clutter. This division lets the model exploit global context while preserving fine hand articulation.
In experiments across five datasets and four video backbones, DreamHMR outperformed existing methods on all reported metrics for 3DPW, EMDB-1, and RICH, cutting body MPJPE by 4.2%, 4.7%, and 6.1% respectively. On held-out occlusion-heavy sequences from BEDLAM and EgoBody, overall body and hand MPJPE dropped by 7.0% and 10.0%. The authors plan to release code publicly, which should help others apply the approach to downstream motion-understanding tasks.