A new benchmark called LEAPS, described in an arXiv preprint, aims to evaluate long-context language models on adaptive pattern learning and state tracking. The paper's abstract frames these abilities as central to the growing use of long-context models in agentic settings.
Long-context models are increasingly deployed as agents that act over extended trajectories, from improving a codebase to running research experiments to forecasters that condition on lengthy histories. According to the abstract, these applications draw on the capabilities that LEAPS is designed to assess.
The provided source is limited to the abstract, which is truncated mid-sentence, so details on the benchmark's construction, tasks, and results are not available from the text. The announcement type is "new," indicating a fresh preprint rather than a revision.