Reinforcement learning offers a way to align diffusion-based visual generators with reward signals that go beyond their original pretraining goals. According to a new paper on arXiv, current score-function methods typically achieve this by injecting stochasticity through perturbations of the reverse denoising trajectory. The paper argues that this approach has limitations, though the abstract does not specify them in detail.
The authors propose an alternative: routing policy optimization for online diffusion reinforcement learning. Rather than perturbing the denoising path, the method optimizes a routing policy that determines how the diffusion process proceeds. This shift in focus could offer a more direct way to maximize reward signals while generating images.
Because only the abstract is available, the article does not include experimental results or comparisons. The source is a single preprint, so there are no differing viewpoints to note. The significance lies in the conceptual move from trajectory perturbation to policy-based routing in diffusion RL.