Budgeted language-model cascades are a common way to control costs by routing queries through increasingly capable models. But auditing such systems is difficult: you need to know which calls were actually made, which service or cache produced each result, and whether the recorded execution matches reality. CascadeLedger, a new framework from arXiv, addresses this by introducing counterfactual execution auditing.

The protocol enforces call-before-read semantics, meaning a read is only considered valid if the corresponding call was actually executed. It also binds successful reads to the identities of the service and cache that supplied them, creating a clear chain of accountability. The framework records this information as part of the audit trail, allowing later verification of both information integrity and resource usage.

Because the auditing is counterfactual, it can check not only what happened but also what should have happened under the stated budget and routing rules. This makes it possible to detect mismatches between claimed and actual execution, which is useful for debugging, compliance, and ensuring that cost-saving cascades are not silently skipping required steps. The approach is still at the research stage, but it points toward more transparent and trustworthy LLM pipeline operations.