Open-vocabulary segmentation often works by lifting 2D semantic features into a 3D scene representation. According to the OV-Foam preprint, current approaches typically rely on simple backprojection to assign those features to geometric primitives, treating each primitive independently during the transfer.
The paper focuses on 3D Gaussian splatting, a representation where primitives overlap substantially. The authors note that this heavy overlap is a key source of difficulty for naive backprojection, because a single 2D feature can correspond to multiple, mutually occluding primitives, making the assignment ambiguous.
By framing the problem around how the 3D representation itself shapes feature lifting, OV-Foam highlights that the success of open-vocabulary segmentation depends not only on the power of the image encoder, but also on the geometric structure into which features are placed. The abstract stops short of giving full implementation details, but the paper's stated motivation is to move beyond one-size-fits-all backprojection for overlapping representations.