Transformer compression typically treats attention capacity and the number of layers that execute it as a single trade-off: prune attention, and you also reduce depth. A new arXiv preprint argues that these two quantities may be separable, at least in hierarchical vision transformers such as SegFormer. The authors ask whether useful attention functions can be relocated rather than simply removed.
To explore this, they introduce the idea of path-conditioned attention mobility. Rather than assuming each layer must keep its own attention budget, the work considers how attention capacity could be concentrated in specific 'execution hubs' while other depths are bypassed. The framing shifts the question from 'how much attention to keep' to 'where in the network attention should run.'
The paper is an analysis of compression design rather than a new state-of-the-art result. Its contribution is conceptual: it exposes an assumption hidden in most pruning and distillation methods. If attention capacity and depth are indeed independent, future compression schemes could be built around mobility rather than uniform reduction.