Dimensionality reduction is often used to shrink text embeddings, and a new preprint on arXiv reports that average performance can survive even when a substantial fraction of dimensions is removed. That sounds reassuring, but the paper cautions that aggregate metrics do not show whether the damage is distributed evenly across individual examples.

The authors propose that geometric structure can predict per-example neighborhood loss under PCA truncation. In other words, the local arrangement of embeddings before truncation may indicate which items are most likely to lose their nearest neighbors afterward.

This matters because applications that rely on embedding neighborhoods—such as retrieval or clustering—may be silently degraded for specific inputs even when overall benchmarks look fine. The finding is based on a single arXiv abstract, so no independent verification or conflicting results are available yet, but it points toward a more granular way to evaluate dimensionality reduction than average scores alone.