The paper introduces FocusOPD, a structure-guided on-policy distillation approach for diffusion models. It is designed for multi-teacher distillation, where several task-specific models are merged into one student model. The method applies to rectified-flow models with diffusion-transformer architectures.
The abstract notes that while multi-teacher on-policy distillation can combine capabilities, matching model outputs does not guarantee effective distillation. This limitation motivates the structure-guided strategy, though the abstract does not specify the exact structural cues used or report experimental outcomes.
As an arXiv announcement, the paper is new and not yet peer-reviewed. Further details on the method's implementation and results are not available in the abstract.