This paper, posted on arXiv, studies in-context binary classification in a deliberately simple attention model. The model is trained to minimise a logistic loss, and each prompt is generated by a latent logistic teacher. The learner must infer the teacher's direction from context examples whose labels are drawn as Bernoulli variables.
The work focuses on calibration and noise in this setting. By using linear attention with a softmax readout, the model keeps the architecture tractable while still performing classification from the labeled examples in its context. The abstract does not state the main results, so the specific conclusions about calibration and noise are not available from this summary alone.
The significance of the paper is in providing a theoretical framework for understanding how attention-based models learn from in-context examples, and how that learning behaves under noisy conditions. Further details would require reading the full manuscript.