When a 3D scene-grounded vision-language model (VLM) has a limited context window, it cannot attend to every object and relation in a scene. A new arXiv preprint (2610.14207) frames this as a candidate-retrieval problem: under a fixed token budget, which objects and relations should the model see first?
The authors report results on the ScanNet indoor referring benchmarks. Matching the query to object category names outperforms using pairwise geometry for candidate retrieval. This suggests that, at least for referring tasks in indoor scenes, semantic category labels are a more efficient first-pass filter than geometric relationships.
Because the abstract is cut off, the full extent of the comparison—such as how much the gap is or how geometry performs in other settings—is not yet clear. The paper's central question, however, is timely as 3D VLMs grow in capability while context budgets remain a practical constraint.