When a video editing tool removes an object from the frames, the object's sound often remains in the soundtrack, creating an obvious mismatch. According to a new arXiv preprint, existing video inpainting models operate solely on pixels and therefore ignore this acoustic trace. The paper introduces TV-AudioRemover, a method designed to jointly remove both the visual object and its corresponding sound.

The approach is guided by text and visual information, allowing the model to identify which part of the audio belongs to the removed object. The authors also propose a multi-task hard-mixture curriculum to train the system, presumably balancing the difficulty of visual and audio removal tasks. Because the abstract is truncated, details about the architecture and evaluation are not available in the source.

This is a single preprint, so there is no independent verification or comparison with other methods. Still, the work highlights a growing focus on audio-visual consistency in video editing, rather than treating sound and vision as separate problems.