Mixture-of-experts (MoE) models can be made cheaper to serve by sending prompt tokens through fewer routed experts during prefill. This simple cost-saving trick is attractive, but its side effects have been unclear. A new preprint on arXiv sets out to locate exactly what this approximation damages.
The authors find that the harm is concentrated: only a few prompt tokens are responsible for most of the short-answer F1 loss. Using token-, layer-, and attention-path-level analysis, they show that the degradation is not spread evenly across the input, but tied to specific tokens that disproportionately affect the model's final answer quality.
The result is a step toward understanding when prefill expert reduction is safe and when it is not. Because the paper is a preprint and the abstract is brief, the full methodology and experimental details remain to be examined, but the finding offers a concrete target for future serving optimizations that want to cut cost without silently sacrificing accuracy.