Learned world models let planners simulate candidate actions, but a planner can improve its predicted outcome while actually choosing a worse action. A new arXiv paper dissects expected free energy (EFE), an objective that combines goal distance with disagreement among stochastic predictions, as a way to control how much test-time computation a latent world-model planner should spend.
The authors tested candidate scoring, stopping rules, and online predictor updates separately. A stopping rule that halts refinement when elite cost stalls matched fixed-depth references on tasks whose success plateaus with depth. The disagreement term was less consistent: with a frozen model and larger weights, rewarding disagreement lowered success, while penalizing it helped in some task and weight settings but hurt in others.
Online predictor updates recovered part of the loss when training data omitted a region needed to reach the goal. The tested cost-stall rule can still fail when further optimization exploits model errors, because predicted cost continues falling after actual task success has peaked. The study concludes that cost progress can usefully guide computation on a plateau, but protecting performance after that peak requires information about the consequences of the chosen actions.