Continuous video and electronic clinical records capture different aspects of a patient's time in hospital. Video shows moment-to-moment movement and interaction, while records provide structured context about condition and care. A new preprint on arXiv introduces TRACER, a multimodal harness designed to bring these streams together for a unified view of patient behavior.
The central problem the authors address is representation: how to model behavior across the entire stay rather than in isolated snapshots. TRACER's design as a harness suggests a wrapping or integration layer that aligns the two modalities over time. According to the abstract, the key is preserving complementary information from both video and clinical data while building a stay-level behavior representation.
The work is at the preprint stage and has not been peer-reviewed. Further details on the architecture and evaluation are not yet available in the abstract, but the framing points to a practical challenge in hospital AI: making sense of the rich, messy data already being collected at the bedside.