Large language model agents often solve tasks by generating long sequences of tool calls, and the standard design uses a single strong model for every step. But many of those steps are routine enough that a much smaller model could handle them, according to a new paper posted on arXiv.

The paper, titled "Trajectory-Level Speculation with Semantic Verification for Tool-Using Agents," suggests a speculative execution approach. A small model drafts entire trajectories of tool calls, and a stronger model then checks those drafts semantically rather than generating each step from scratch.

This could lower the computational cost and latency of tool-using agents, though the abstract does not include experimental results. The full details of the method and its evaluation are available in the arXiv preprint 2610.21463.