Group relative policy optimization (GRPO) is a common method for RL post-training of language models. It samples a group of responses to a prompt and updates the policy only when the rewards in that group disagree. With a binary verifier, a group that is entirely correct or entirely incorrect has zero within-group variance, so the method receives no learning signal.
The paper introduces a probe prior that explicitly models nonzero within-group variance, aiming to preserve a learning signal when the binary verifier returns the same outcome for all responses. This is framed as a way to address the degenerate case where the usual variance-based update would stall.
The work is relevant to RL post-training pipelines that rely on binary correctness signals, where homogeneous groups are common. By treating zero variance as a prior rather than a stopping condition, the approach could make GRPO more robust. The source is a single preprint; no comparison to other methods is included in the available abstract.