A robot that plans while moving faces a classic dilemma: how long should it let the planner think before committing to an action? Longer deliberation can yield a better plan, but it also delays feedback from the world. Most controllers fix this computation budget before deployment, which the authors of a new arXiv preprint argue is a mistake. The optimal planning time is state-dependent, varying with unmodeled dynamics, safety constraints, and the diminishing returns of further optimisation.

To address this, the researchers cast the problem as a semi-Markov decision process (SMDP), where a single decision sets both the robot's action commitment and the time allocated to the solver. They then train a reinforcement-learning policy that takes as input the current state, the executing plan, and inexpensive planner statistics. The policy learns to adjust the budget dynamically rather than relying on a static setting.

The approach was evaluated across three very different domains: ground-robot navigation, agile quadrotor flight, and contact-rich manipulation. In each case, the learned policy changed its computation budget according to the demands of the scenario and outperformed fixed-budget baselines. The results suggest that metareasoning—deciding how much to reason—can itself be learned, and that adaptive budgets may be a practical way to balance deliberation and reactivity in real-time control.