Reinforcement learning (RL) can become off-policy when training conditions diverge from inference, through mini-batch updates, asynchronous training, or a mismatch between training and inference. A new arXiv preprint argues that this mismatch is especially harmful in Mixture-of-Experts (MoE) models, where routing decisions are central to performance.

The paper proposes a method called Router Drift Control to mitigate this degradation. The abstract does not specify the mechanism, but the name points to the router—the component that assigns inputs to experts—as the focus of the intervention.

Because only the abstract is available, no experimental results or comparisons are included here. The contribution is framed as a control approach for off-policy drift in MoE reinforcement learning, rather than a full empirical study.