Automating synthesis planning for small-molecule drug design demands AI models that can predict reaction outcomes reliably. Yet frontier language models still fall short, and a new arXiv paper suggests that some popular training techniques may be making the problem worse.

The paper, titled "Distillation, Self-Training, and the Fragility of Learned Chemical Reasoning," examines how distillation and self-training affect the robustness of chemical reasoners. These methods are often used to compress or adapt models, but the study indicates they can produce fragile reasoning—models that seem competent on familiar cases but fail unpredictably on new ones.

The authors argue that chemical reasoners need to be open to inspection and adaptation, not just accurate on benchmarks. The fragility identified in the study implies that chasing performance through distillation or self-training may sacrifice the very robustness that real-world drug design requires.