Low-rank adaptation (LoRA) is a popular method for fine-tuning large models, but it treats all tokens uniformly: every rank-one component of the update is applied to every token. A new arXiv paper (2610.19033) contrasts this with mixture-of-experts variants, which use a learned router to activate a fixed number of components per token.
The paper's central argument is introduced but cut off in the abstract: "We argue that the number of…" Based on the title, the authors propose a noise-floor squelch for rank-one LoRA experts, suggesting that some components may be unnecessary for certain tokens and can be suppressed when their contribution falls below a noise floor.
Because the abstract is truncated, the full method and results are not available from this source. The key contribution appears to be a principled way to decide how many LoRA experts to activate, rather than using a fixed count or always using all of them. Further details would require reading the full paper.