Pre-trained speech models build rich internal representations that blend many properties of an utterance—pitch, loudness, syntax, and meaning—into a distributed encoding. This entanglement is a problem for interpretability: when a model performs a downstream task, it is hard to tell whether it is relying on what was said or how it was said. The authors of a new paper, accepted to Transactions of ACL, propose a way to pull those strands apart after the fact.

The method applies the Information Bottleneck principle to representations taken from a pre-trained speech encoder. It works in a cascaded fashion: first, it learns a vector that captures only the content—the part that can be transcribed as text—and nothing else. Second, it learns a separate vector for all complementary acoustic features that are relevant to a specific target task. This post-hoc separation means the original encoder does not need to be retrained.

The authors tested the approach on emotion recognition and speaker identification. By disentangling the representations, they could quantify how much textual versus acoustic information contributes at each layer of the model. They also used the method for feature attribution, identifying which speech frames are most salient from a textual perspective and which from an acoustic one. This offers a practical tool for auditing and understanding speech models, though the paper focuses on two tasks and does not claim generalisation to all downstream uses.