Novel view synthesis methods like NeRFs and 3D Gaussian splatting depend on accurate camera poses, typically obtained through Structure-from-Motion (SfM). Classical SfM pipelines use local feature matching and bundle adjustment, which is robust but computationally heavy. Recent learning-based approaches train transformer networks to regress poses directly, offering large speedups at the cost of accuracy.
The authors benchmark these strategies on existing datasets, testing two simple efficiency improvements. First, they show that reducing the number of features used in classical SfM already provides a significant speedup while maintaining high pose accuracy. Second, they find that using a feed-forward network to generate initial pose estimates and then refining them with classical SfM techniques achieves the best efficiency-effectiveness trade-off overall.
The paper's public benchmark and code are intended to encourage further research on SfM methods that are both fast and accurate. The results suggest that hybrid pipelines, rather than pure learning or pure classical approaches, are the most promising direction for practical novel view synthesis.