Group relative policy optimization (GRPO) improves a model only when the sampled responses to a question disagree. According to a new arXiv paper, a group that is entirely correct or entirely incorrect produces zero reward variance and a zero advantage, so the policy receives no useful learning signal.
The authors argue this is a particular problem for web rollouts, where such uniform groups are common. Their proposed method, ridge-tracking Gaussian sampling, is designed to steer sampling toward mixed groups—those containing both correct and incorrect responses—so that learning can actually occur.
The paper is titled "Follow the Mixed Groups: Ridge-Tracking Gaussian Sampling for Zero-Variance Web Rollouts" and is available as arXiv:2610.25330. The abstract does not yet report experimental results, so the claims are limited to the method's motivation and design.