Language model perplexity has long served as a convenient proxy for estimating automatic speech recognition word error rate, with prior work suggesting a roughly linear relationship in log-log space. A new paper from researchers including Mohammad Zeineldeen and Hermann Ney revisits that assumption for modern end-to-end ASR systems, which embed internal language modeling capacity and are often used without an external LM. The authors test whether external LMs still help, whether the linear relation holds, and how encoder context length and LLM perplexities fit into the picture.

The study finds that external LMs do still improve current end-to-end ASR, but the perplexity–WER relation is no longer consistently linear in log-log space. The authors show that encoder context length affects the relationship, and that subtracting the decoder's internal language model—a technique known as ILM subtraction—changes the observed trend. This indicates that the decoder's internal LM must be accounted for when interpreting external LM quality.

The paper also examines how LLM perplexities compare with standard neural LM perplexities, finding that they do not simply fall on the same trend line. The results suggest that perplexity should be used with caution as a proxy for WER in modern systems, and that the internal language modeling behavior of end-to-end models is a key factor in any such comparison. The work was accepted at IEEE SLT 2026 and is available on arXiv.