Multimodal sentiment models are typically evaluated on complete utterances, but real-world streams often lose a contiguous chunk of text, audio, or video. The authors propose RM-Net, which keeps the original temporal positions, represents unobserved positions with a learned token, and gates per-modality intensity contributions by observation quality. Training uses dual-view masking of contiguous segments.
On aligned CMU-MOSEI features, the fused model—five RM-Net seeds plus a LoRA-adapted BERT text expert—reaches a macro-F1 of 0.655 on clean inputs and 0.634 under 40% mixed missingness. Its two-view average exceeds an auxiliary-head baseline by 0.024, with a 95% video-cluster bootstrap interval of [0.003, 0.045]. The authors explicitly note that the held-out split was inspected during development, so these numbers are not a blind test.
For interpretability, exact Shapley attributions over the eight modality subsets and position-level occlusion show that top-ranked positions reduce predicted-class probability faster than random positions (AUC gap 0.113 on validation). The paper also describes an auditable word-level alignment pipeline for raw clips, but its outputs do not feed the predictive models.