Multimodal objects—such as products in an e-commerce catalog—can be grouped in multiple valid ways depending on the task. A shopper might want to cluster items by color, by material, or by both simultaneously. The authors of a new arXiv preprint argue that embeddings trained for retrieval are not well suited to recovering these arbitrary partitions, because they are optimized for similarity search rather than for flexible grouping.
The paper introduces CIM-Embed, a method for learning instruction-conditioned multimodal embeddings. The core idea is that the embedding function takes a natural-language instruction as input, so the resulting clustering adapts to the specified criterion. This allows the same dataset to be partitioned in different ways without retraining the model for each new grouping.
The abstract highlights the limitation of retrieval-oriented embeddings but does not provide details on the model architecture or experimental results. As a preprint announcement, the source offers only a high-level description of the proposed approach and its intended use case.