Reinforcement learning that maximizes expected reward often collapses onto a small set of outcomes, even when tasks require many diverse, high-quality solutions. This phenomenon, known as mode collapse, is the focus of a new arXiv paper.
The paper proposes MIPS (Mitigating Mode Collapse in Reinforcement Learning for Large Outcome Spaces). According to the abstract, the method uses inverse probability scaling to address the issue, though the abstract does not provide further technical details.
The work is relevant for applications where a range of distinct solutions is needed, such as creative tasks or multi-solution problem solving. The abstract does not include experimental results or comparisons, so the effectiveness of MIPS remains to be assessed from the full paper.