Generalist vision-language-action (VLA) policies can transfer to new scenes with little task-specific training, but they often fail silently: the action stream alone does not reveal whether a rollout is collapsing or making progress. A terminal success/failure bit is too sparse to say when things went wrong. The authors of a new arXiv preprint argue that the per-frame label is the hard part, and that a continuous, non-monotonic score—one that can fall and rise again as a rollout goes wrong and recovers—works better than a binary label.
To get such labels cheaply, a person marks only the stage at which an episode went wrong; per-frame values are then computed from that annotation. The resulting model, ValueFormer, is a policy-agnostic causal transformer with 3.5 million parameters on a frozen DINOv3 backbone. It emits two per-frame signals in one forward pass: a smooth Monte Carlo value for advantage estimation and a sharp binary value for online mistake detection. Failed episodes carry a stage-aware return that rises before the marked stage and decays afterward.
The authors show that reusing an intervention flag already recorded at a teleoperation station lifts held-out detection from 0.38 to 0.82 average precision and flags 95% of mistakes before the operator reacts. On a real bimanual sandwich station, feeding the critic back as a per-frame post-training weight raised task completion from 70% to 85% and removed a repeat-pick failure mode in their trials. The authors caution that at n=20 that gain sits within noise.