A new arXiv paper tackles multi-teacher on-policy distillation (MOPD), a setup in which several domain-specialist policies are combined into a single shared student. The central difficulty is deciding how strongly each teacher should supervise the student during training.
The authors argue that existing discrepancy-based allocation, which weights teachers according to how much their policies differ from the student's, can assign influence poorly. They propose moving beyond policy discrepancy to a capability-aware allocation that accounts for what each teacher can actually contribute.
The provided abstract is truncated, so the full method and experimental results are not described in the available text. What is clear is the paper's direction: replacing a purely discrepancy-driven weighting scheme with one that is sensitive to teacher capability.