The paper argues that as frontier LLMs become more capable, their running costs increase. In contrast, small language models (SLMs) have improved while keeping costs relatively stable, offering 5-40 times cost savings.
The proposed approach is to use a large LLM for iterative development and experimentation, then serve the final system on an SLM. This separates the development phase from deployment, allowing teams to benefit from the LLM's capabilities without paying its inference costs at scale.
The abstract is brief and does not provide implementation details or benchmarks. It suggests a practical strategy for organizations looking to balance capability and cost.