Reinforcement learning keeps improving large language model agents in verifiable settings, but two new arXiv papers highlight why it has not yet translated into broad, flexible competence. The first, ArenaFlow, focuses on open-ended agent tasks, where solutions are diverse and reliable scalar rewards are hard to obtain. It proposes moving from trajectory ranking to hierarchical credit propagation to address this challenge.
The second paper, on consolidating RLVR capabilities, starts from a similar observation: RL with verifiable rewards improves specific capabilities, but covering multiple capabilities often means training separate domain experts and then consolidating them. The paper takes a deep dive into fusion paradigms for that consolidation step.
Where the two sources agree is in treating RL as a strong but narrow tool. Where they differ is in what they see as the next problem to solve: ArenaFlow targets credit assignment within a single open-ended task, while the fusion paper targets integration across separately trained experts. Read together, they point to two complementary research fronts—better learning signals inside one task and better combination of skills across tasks.