Reasoning over long document contexts remains a challenge for large language models, according to a new arXiv paper. The authors argue that standard retrieval methods are inherently reactive, creating a bottleneck for agents that need to perform complex cross-document reasoning at inference time.

To address this, the paper introduces 'Librarian Agents' — LLMs trained to curate long-context workspaces. Instead of waiting for a query and then retrieving relevant passages, these agents build and maintain a workspace that anticipates future queries, making relevant information available ahead of time.

The proposal shifts the focus from reactive retrieval to proactive curation. If successful, this could reduce the inference-time burden on agents handling long documents, though the abstract does not yet include experimental results or implementation details.