Large multimodal models (LMMs) must understand how entities move and interact as a scene unfolds. The authors of a new arXiv preprint argue that this requires tracking the same entities over time and relating their observed motion to later changes in the scene.
To address this, they introduce SceneForesee, a framework for learning scene evolution. The abstract frames this as a spatiotemporal reasoning problem, though the available text stops short of describing the model architecture or training procedure.
The work highlights a key limitation in current LMMs: they often treat scenes as static snapshots rather than dynamic processes. By explicitly modeling how entities move and interact, SceneForesee aims to improve reasoning about what happens next.