Reinforcement learning has made strides in domains like mathematics and code, where verifiable rewards give clear feedback. But many valuable open-ended tasks do not fit this pattern. According to a new arXiv paper, these tasks share three properties: no automatic verifier of overall quality, multiple requirements that must be satisfied jointly, and—the abstract begins to explain—a need for a different reward mechanism.
The paper introduces RARE, or Reinforcement Learning with Adaptive Rubric Evolution. Rather than relying on a fixed, verifiable reward, RARE evolves a rubric over time to guide learning on open-ended generation. The title suggests the rubric itself adapts, which could help models balance competing requirements without a single ground-truth score.
The abstract is brief, so details of the rubric-evolution mechanism are not fully described in the source. What is clear is the motivation: extending RL beyond closed, verifiable tasks to open-ended ones where quality is harder to define and multiple objectives must be met at once.