SpeechLLMs typically recognise emotion by generating a label token, relying on the model's existing output representations. Researchers instead tested a discriminative adaptation: a linear classification head on top of the LLM's hidden states. Using a compact speechLLM (frozen Whisper-medium encoder with a 3B LLM) on IEMOCAP and MELD, they compared both approaches under two transcript conditions: ground-truth text and automatic speech recognition (ASR) output.

The classification head improved Macro F1 in 10 of 14 configurations. On IEMOCAP, it improved every configuration that used ASR transcripts, reaching 76.97% Macro F1, with a significant gain when using speech alone. On MELD, where ASR transcripts had a 32.38% word error rate, the head helped with speech alone but not with ASR transcripts.

The benefit depended on both dataset and transcript source. In 4 of the 6 configurations that used text, the head's effect changed sign between ground-truth and ASR transcripts, meaning conclusions drawn with ground-truth transcripts do not necessarily hold in deployed ASR settings. The authors also found that the learned class vectors' nearest tokens were mostly emotion-related words rather than explicit emotion labels, offering a clue to how the discriminative head encodes emotion.