Mixture-of-Experts language models save computation by activating only a subset of expert parameters for each token. Yet deployment remains heavy because every expert must be stored, even when rarely used. A new arXiv preprint examines post-training expert pruning as a way to shrink this footprint.
The study frames pruning as a reconstruction problem, using task-domain calibration to decide which experts can be removed with least damage to the model's outputs. This is a post-training approach, meaning it does not require the original training pipeline or a full retraining pass.
The paper is a preprint and has not been peer-reviewed. It reports on the method's design and evaluation, but the abstract alone does not provide specific numerical results or comparisons. As with other pruning work, the key trade-off is between storage savings and model quality after removal.