Foundation models need to support diverse inference budgets, yet training a separate model for each budget duplicates pretraining effort. Elastic models address this by training several nested sub-networks inside one weight-shared supernet, so a single run can yield models of different sizes.
The new preprint ShellElastic takes this idea in a specific direction: its title promises scalable elastic pretraining with simultaneous maximal updates and hyperparameter transfer. That combination is the paper's stated contribution, distinguishing it from earlier elastic-training work.
A caveat: the abstract provided is truncated mid-sentence, so the available text does not say what limitation of prior elastic methods ShellElastic overcomes, nor does it report results. The digest above reflects only the problem framing and the proposed technique named in the title.