Long chain-of-thought (CoT) distillation has become a standard technique for improving reasoning in language models, yet the reasons for its success remain unclear. A new preprint on arXiv, titled "Facts Don't Speak Louder than Words: The Behavioral Essence of Long-CoT Distillation," takes up this question. The authors point out that current pipelines typically rely on rejection sampling to gather training data, but the behavioral consequences of this approach are not well understood.

According to the abstract, the study aims to uncover the underlying mechanism rather than simply measure performance gains. The title hints that the content of the distilled reasoning traces—the "facts"—may be less important than the behavioral patterns they induce in the student model. This suggests a shift in focus from what the model knows to how it behaves.

Given the limited abstract, the full findings are not yet available, but the framing raises a key question for the field: if behavior, not factual content, drives improvements, then distillation strategies might need to be redesigned around behavioral objectives rather than answer correctness.