Modern LLM applications increasingly rely on runtime monitors that watch tool calls, traces, streams, detector outputs, and verifier inputs. These monitors are meant to provide safety evidence, but the paper argues that the interfaces connecting them to the framework create a reliability problem.
The key issue is that a framework path may remain operational while its validity as evidence degrades. In other words, the monitor can keep running and producing outputs, but those outputs may no longer reflect what the system actually did or decided. The paper calls these "monitor-validity failures."
This is a subtle failure mode because it is not a crash or an obvious error. The monitor appears healthy, so teams may trust its signals even as the underlying framework evolves and the evidence path becomes invalid. The abstract suggests that these failures occur at evolving framework boundaries, meaning they are likely to become more common as LLM applications change over time.