Large audio-language models (LALMs) can identify acoustic attributes such as emotion, speaker gender, and spoken language, even though their language-model backbones are pretrained on text alone. How audio representations connect to text before the model produces an answer has remained unclear. A new preprint posted to arXiv tests the idea that audio and text representations meet in a shared subspace within the network.
The authors located "cross-modal neurons" that respond to both audio and text. They found alignment across three LALMs and four acoustic attribute types, though to different degrees. This suggests the neurons act as a bridge between modalities rather than as separate processing streams.
To test whether these neurons actually drive behavior, the researchers performed causal interventions. Steering or deactivating the neurons changed model predictions, indicating they are not merely correlated with acoustic recognition but are functionally required and can be manipulated. That points toward a mechanism for controlling LALM behavior at inference time, though the paper does not claim a full mapping of the alignment subspace.