In-context learning (ICL) lets language models adapt to new tasks from a few examples, but this ability can be costly at inference time. A new arXiv paper investigates whether ICL can be compressed into latent objects—such as task vectors, function vectors, or context vectors—that would deliver few-shot performance with zero-shot efficiency. The authors acknowledge that such amortization works in many cases, but they identify a key theoretical boundary.
The paper's title states the central finding: "Rules Amortize, Pairings Don't." According to the abstract, recent theory shows that a static vector acts as a single... (the text is cut off). The title and framing suggest that tasks with rule-like linguistic structure can be captured by a static representation, while tasks that depend on arbitrary pairings between inputs and outputs cannot. In other words, the structure of the task determines whether a latent vector can replace in-context learning.
This distinction has practical implications for when amortized models are reliable. If a task relies on memorized pairings rather than generalizable rules, a static vector may be insufficient, and true in-context processing may be necessary. The paper thus offers a theoretical lens for deciding when zero-shot approximations of few-shot behavior are safe to use. Further details are limited because the provided abstract is truncated, but the core claim is clear: not all ICL is equally amortizable.