Joint audio scene classification and audio tagging are often handled by multihead models that treat the two tasks as independent. This paper explores a hierarchical alternative: instead of separate output heads, the model predicts scene categories directly from intermediate event probabilities, reflecting the natural dependency between sounds and scenes.
The authors benchmarked this approach with 66 full training runs across three state-of-the-art backbones and various loss-weighting settings. Hierarchical conditioning achieved classification performance competitive with multihead baselines, but it changed the optimization dynamics. While multihead models tended to solve the easier scene task before the tagging task, hierarchical models allowed synchronized dual-task learning when the loss weight beta was between 0.6 and 0.9.
To understand why, the authors introduced a multi-perspective interpretability framework combining information bottleneck analysis, manifold geometry, and Formal Concept Analysis on a TF-IDF event representation. Hierarchical conditioning consistently produced lower conditional entropy, higher mutual information, and higher Silhouette scores, indicating a more structured and semantically faithful intermediate feature space.