Chain-of-thought (CoT) prompting has become a standard technique for improving transformer reasoning, but the mechanisms behind its success remain poorly understood. A new paper on arXiv takes a step toward closing that gap by analyzing a minimal setting: a one-layer transformer trained to perform in-context K-nearest neighbor (KNN) prediction. The authors provide a proof that such a model can learn this task when CoT is used, offering one of the first rigorous accounts of CoT's role in training.

Specifically, the paper shows that the transformer can generate intermediate reasoning steps that correspond to the KNN algorithm's computations, and that these steps are learnable from data. The proof is notable because it does not rely on deep architectures or complex attention patterns; a single layer suffices. This makes the result a clean testbed for understanding how CoT enables otherwise difficult tasks to become tractable for simple models.

The significance is twofold. First, it gives a concrete example of a task where CoT provably expands the capabilities of a fixed architecture. Second, it suggests that the emergence of reasoning during training may be understood through the lens of algorithmic alignment, where the model learns to simulate a known procedure step by step. The authors do not claim that this toy setting explains all of CoT's empirical success, but it provides a foundation for more general theories.