Proactive dialogue agents face a tricky challenge: they must guide multi-turn conversations toward a task goal while responding to users whose intentions and attitudes shift over time. Existing planners often rely on search or learned policies, but these can struggle when users behave unpredictably. A new paper on arXiv proposes a training method called Mentalized Adversarial Self-Play, which addresses this by simulating users that are deliberately 'soft'—meaning they are not rigidly fixed in their behavior.

The key idea is to train the agent against these simulated users in an adversarial self-play setup. Rather than optimizing against a single, static user model, the agent learns to handle a range of flexible responses. This is intended to make the agent's action selection more robust in real interactions, where users may change their minds, resist suggestions, or become more cooperative over time.

The paper's abstract emphasizes that this improves system-side action selection, but the full details are not yet available in the abstract alone. As a single source, this is an early-stage research claim, not a proven result. Still, the approach points toward a promising direction: using mentalized models of users to train agents that can proactively achieve goals without assuming users will behave like simple test cases.