Third-party LLM gateways act as intermediaries, choosing which providers, models, endpoints, and fallback paths to use for each inference request. They also generate the records that explain and bill those choices, meaning the same entity both makes the decision and writes the official account of it.
A new arXiv paper introduces evidence-bound gateway-path provenance, a method designed to make these records more trustworthy. The abstract is truncated, so the full technical details are not available, but the core idea is to bind provenance to evidence, presumably allowing users to verify that the recorded path matches what actually happened.
This matters because when a gateway is both decision-maker and record-keeper, there is a risk that explanations or bills could be inaccurate or self-serving. Evidence-bound provenance could provide a way to audit the gateway's claims, though the paper's specific mechanisms remain unclear from the abstract alone.