End-to-end speech large language models frequently answer the same question less accurately when it is spoken rather than typed, even when the spoken content matches the text prompt. A new preprint on arXiv proposes a layer-wise analysis of this speech-text modality gap, arguing that existing studies treat the gap too coarsely.

Rather than assuming that aligned representations are functionally equivalent, the authors distinguish geometric alignment from functional equivalence. The work suggests that representations can line up geometrically across modalities while still failing to support the same downstream performance.

Because the abstract is truncated, the specific method and results are not fully detailed here. The significance lies in reframing the modality gap as a layer-wise, functional problem rather than a single embedding-space mismatch.