Reinforcement learning (RL) has become a standard approach for post-training large language models, according to a new arXiv paper. The abstract notes that in typical implementations, learning is off-policy: the policy that generates responses differs from the policy being trained. The reason for this difference is cut off in the source, but the statement sets the context for the paper's investigation.

The paper is titled "Baselines and Score Centering in Policy Gradient Estimation." While the full abstract is not available in the source, the title suggests the work addresses how baselines and score centering—techniques used to reduce variance in policy gradient methods—apply in this off-policy setting. The source does not provide specific results or conclusions.

Because the abstract is truncated, this digest can only report the opening context. Readers interested in the full findings should consult the arXiv listing.