Geometry foundation models that estimate camera poses and 3D structure from video typically process frames in fixed windows, aligning predictions across windows. Errors accumulate over long sequences, and streaming variants that use past frames cannot correct earlier estimates once new evidence arrives.
The proposed framework, SMORE (Scaffold MemOry for Revision), attaches a revisable external memory to a frozen geometry model. Inspired by hippocampal associative memory, it links spatial location ('where') with observations ('what'), allowing past estimates to be stored, retrieved, checked against new evidence, and revised when inconsistencies appear.
The authors tested SMORE with eight competitive geometry models on five datasets covering pose estimation, multi-view depth, point-cloud reconstruction, SLAM, and novel view synthesis. Across all models, it improved long-sequence pose estimation, reduced multi-view depth error by up to 58%, cut Chamfer distance for point-cloud reconstruction by up to 43%, and achieved average trajectory error as low as 2.9 cm in long-horizon SLAM.
The results suggest that revisable external memory can act as a general interface for extending pretrained geometry models beyond fixed-context prediction, enabling iterative, error-correcting inference over long visual sequences.