Most recent work on deriving text embeddings directly from frozen large language models has been dominated by English, leaving multilingual performance largely unexamined. This arXiv preprint takes a step toward closing that gap by proposing a method based on paired-residual subspace removal.
According to the abstract, the technique is designed to work in a training-free setting, meaning it does not require fine-tuning the model. Instead, it operates on the residual subspaces produced by the frozen model, steering the embedding space to better handle non-English text. The exact details of the algorithm and its experimental results are not included in the available source text.
Because the source is a single preprint abstract, independent verification and comparison with other multilingual embedding methods will require reading the full paper. The work is notable for explicitly targeting a known blind spot in the training-free embedding literature.