A single arXiv preprint, Proxy Confidence: Auditing Black-Box LLM Agents with a Surrogate's Log-Probabilities, addresses a practical problem: a deployed LLM agent emits tool calls, queries, and code that can be silently wrong. By the time the error surfaces, the action has already run, so catching problems early requires some way to assess the agent's confidence in each step.
The authors note that frontier chat APIs hide the model's token probabilities, leaving the agent's stated confidence as the only direct signal. The paper proposes an alternative: use a surrogate model's log-probabilities as a proxy confidence measure to audit the black-box agent. This would allow external monitoring without access to the original model's internals.
Because this is a preprint and the abstract provides no experimental results, the proposal should be treated as an early idea rather than a validated method. Still, it points to a growing concern: as agents act autonomously, reliable confidence signals are needed to prevent silent failures.