In reinforcement learning, a common technique called GRPO (Group Relative Policy Optimization) updates a model by comparing the relative success of different rollouts. But when every rollout for a given problem fails, the group-relative advantage is zero across the board, and there is no successful exemplar to imitate. As a result, the model receives no learning signal from precisely the problems it needs to learn from most.
The new approach, called Grounding Hindsight, aims to fix this blind spot. Rather than discarding failed rollouts, it recovers supervision from the reasoning traces themselves, turning failures into a usable training signal. The abstract notes that the failed reasoning can be leveraged, though the full mechanism is not detailed in the available excerpt.
This matters because hard problems are often the ones where all attempts fail. If a model never learns from those attempts, it cannot improve on them. Grounding Hindsight offers a way to extract value from those failures, potentially making reinforcement learning more sample-efficient and robust on difficult tasks.