GRPO has become a popular method for reinforcement learning post-training of large language models, but it is not without limitations. Kwai AI's SRPO framework aims to address those limitations using a two-stage reinforcement learning approach with history resampling, according to Synced Review.
SRPO reportedly cuts the number of post-training steps by 90% while matching DeepSeek-R1's performance on math and code tasks. If confirmed, that would make SRPO roughly 10 times more efficient than a standard GRPO-based pipeline.
Synced Review notes the results are presented by Kwai AI and have not yet been independently replicated. The method suggests that significant headroom remains for improving the efficiency of RL-based LLM post-training.