New RL Preprints Focus on Stability, Generalization, and Evaluation

Five new arXiv preprints examine why reinforcement learning remains brittle and what to do about it. Two focus on offline RL: one uses selective reward stimulation for goal-conditioned tasks with sparse rewards and long horizons; another uses boundary-aware data augmentation to reduce overfitting and improve in-distribution generalization. Both aim to make offline-trained policies more reliable, but they target different failure modes—reward signal sparsity versus dataset overfitting.

Two other papers address instability in online RL. One proposes bidirectional behavior-prior distillation to counter systematic critic estimation errors that greedy updates amplify. Another, aimed at RL for large language models, introduces score centering to reduce sensitivity to differences between training and inference engines. They agree that stability is a core obstacle, but they locate the cause in different parts of the learning loop.

The fifth paper takes a different angle: instead of improving training, it improves evaluation. It presents a model-based bootstrap approach for offline policy evaluation in tabular settings, arguing that point estimates are not enough for high-stakes decisions and that principled uncertainty quantification is needed. Taken together, the papers share a common concern with making RL dependable, but they differ on whether the main problem is sparse rewards, overfitting, critic error, inference mismatch, or evaluation uncertainty.