A new paper on arXiv proposes a personalized LLM routing framework that chooses among heterogeneous models with varying costs and capabilities. The router's goal is to minimize cumulative inference cost while ensuring that the quality of responses remains reliably high, even under uncertainty.
The method combines chance-constrained optimization with online preference learning. Instead of assuming a fixed quality threshold, the router learns each user's preferences over time, allowing it to adapt its selections to individual needs while still meeting probabilistic guarantees on response quality.
This approach could help organizations deploy LLMs more economically by sending simpler queries to cheaper models and reserving expensive ones for harder cases, all without sacrificing user satisfaction. The abstract does not provide experimental details, but the formulation points toward a practical way to personalize cost-quality trade-offs at scale.