Building a 3D scene graph means identifying objects and also describing how they relate in space—e.g., “on top of” or “next to.” Existing open-vocabulary methods, which can recognize novel object categories, tend to struggle with spatial predicates. The reason, according to the paper, is that language similarity encodes semantic closeness, not geometric arrangement, so a model that relies on text embeddings alone can confuse relations that look alike in language but differ in 3D space.
The authors propose GHOST, a geometry-constrained hybrid diffusion framework. It explicitly incorporates metric spatial relations as constraints during generation, rather than leaving them to be inferred from language. This hybrid design aims to keep the flexibility of open-vocabulary recognition while grounding the output in actual 3D geometry.
The paper is an arXiv preprint, so the abstract provides limited detail on experimental results. Still, the core idea is clear: for 3D scene understanding, spatial predicates need more than linguistic cues—they need geometric grounding. The work points toward a more robust way to connect language models with physical scene structure.