Semantic segmentation across visual modalities such as 3D point clouds and panoramic images remains challenging, largely because annotated data is scarce and most models are trained for a fixed set of labels. The new work, JOPP-3D, directly targets both problems by framing the task as open-vocabulary segmentation, allowing the model to recognize categories beyond those seen during training.
By jointly considering point clouds and panoramas, the framework aims to exploit complementary information from the two modalities. The open-vocabulary design is intended to reduce the need for exhaustive annotation while improving adaptability to new or rare object classes.
The paper is available on arXiv as a preprint, so its claims have not yet undergone peer review. The abstract does not include quantitative results, meaning the practical effectiveness of JOPP-3D remains to be demonstrated in future evaluations.