Speech-driven 3D facial animation typically relies on self-supervised speech encoders that produce robust acoustic representations. However, as a new arXiv preprint notes, these encoders are not explicitly optimized for the visible articulatory movements—lips, jaw, tongue—that are essential for realistic animation.
The paper introduces CASE, a complementary articulation-aware speech encoder. Rather than replacing general-purpose encoders, CASE is designed to work alongside them, supplying information that is directly aligned with articulatory dynamics. This addresses a mismatch between the pretraining objectives of generic encoders and the requirements of facial animation.
The authors argue that this complementary approach can enhance the quality of speech-driven facial animation by better capturing the relationship between audio and the movements of the mouth and face. The paper is available on arXiv under the identifier 2610.25342.