In hierarchical multi-agent systems, a Leader assigns tasks that subagents execute. Training both levels together is difficult because the final answer gives only indirect feedback to the subagents—they never see a clear signal for their own actions. A new paper on arXiv proposes MultiAgent-OPD, an end-to-end training method that adapts on-policy distillation to this setting.

The core idea is to enable the Leader and subagents to learn from a shared training signal, rather than relying solely on sparse final rewards. The method is designed for systems built around LLMs, where joint optimization often stalls because subagents cannot easily infer which of their choices mattered.

While the abstract does not include experimental results, the contribution is a training framework aimed at a known bottleneck in hierarchical coordination: credit assignment across levels. If effective, this could make it more practical to train complex assistant systems that decompose tasks into smaller agent-run steps.