Reasoning-capable language models frequently generate long chains of thought even for questions that a direct answer could resolve. That wastes inference compute, and while some dual-mode models can respond in either a reasoning or non-reasoning style, the choice is often left to the user. Automating that choice is difficult, according to the authors of a new arXiv preprint, because the routing targets themselves are hard to define.
The paper, titled "CounterRoute: Self-Routed Reasoning via Hierarchical Counterfactual Credit Assignment," proposes a learning-based approach to this routing problem. Rather than relying on an external controller or user preference, the model is trained to route itself, deciding when a reasoning chain is likely to improve the answer. The method uses hierarchical counterfactual credit assignment, a technique for assigning credit to different levels of the routing and reasoning process during training.
The work addresses a practical inefficiency: reasoning can help on complex tasks, but not every query benefits from extended deliberation. If self-routing works as described, it could reduce wasted compute while preserving reasoning capability where it matters. The abstract does not provide experimental details, so the reported approach should be read as a proposal rather than a demonstrated result.