Neural audio codecs are widely used for low-bitrate speech coding and as tokenizers for speech language models, but how their codebook hierarchies represent speaker traits has been unclear. The authors systematically evaluated 17 multi-codebook codecs, reconstructing speech from partial codebooks and using machine listeners to compare trait predictions against original audio. Agreement was measured with reweighted Cohen's kappa, and a "trait depth" proxy captured how many codebook layers are needed to recover each trait.

Results show that trait depth varies substantially: sex has the shallowest depth, followed by speech flow, while voice quality and age group require deeper stacks. Emotion, accent, and non-verbal sounds fall in between. Within traits, speech-flow classes sit at similar depths, whereas emotion, accent, and non-verbal sounds show more variation across classes.

The authors also explored how codec design choices relate to trait depths. Their findings indicate that codebook hierarchies are trait-dependent and may not follow a uniform coarse-to-fine structure, suggesting that future codec evaluation and design should account for which speaker and speech characteristics matter most for a given application.