Modern AI systems increasingly pair large language models with human experts, letting the LLM handle routine queries while deferring uncertain cases for expert review. But when expert capacity is limited, the decision of when to defer becomes nontrivial. A new arXiv paper, "To Defer or Not to Defer? Online Learning with LLMs and Congested Experts," takes up this problem.
According to the abstract, the paper considers the setting where deferral happens under congested expert capacity. The title and framing point to an online learning approach for deciding when to defer, rather than a static rule. The abstract does not provide details on the proposed algorithm or experimental outcomes, so the specific contributions remain unclear from the source alone.