In-context learning (ICL) lets pretrained large language models adapt to new tasks using only a few input-output demonstrations, with no weight updates. But theory has lagged behind practice: most formal accounts treat ICL as a process of identifying a latent task from a known support set, leaving common real-world scenarios unexplained.
The new preprint, "Out of Support, Not Out of Reach," takes on two such scenarios: instructions that fall outside the model's training support, and tasks composed from multiple simpler tasks. The title suggests that being out of support does not make a task unreachable, and the abstract positions the work as an extension of existing ICL theory rather than a replacement.
If the theoretical results hold up, they would help explain why ICL works in settings where the model has never seen the exact instruction before. As with all preprints, the findings have not yet undergone peer review, and the abstract alone does not reveal the full proof strategy or experimental evidence.