Vision Transformers have become a strong choice for semantic segmentation, but their attention mechanism scales quadratically with the number of tokens, which makes them expensive to deploy. A new arXiv paper, SCMerge, tackles this by merging token pairs before the attention computation, with a focus on preserving spatial consistency.
SCMerge builds on a line of recent work that reduces token counts to lower computational cost. The key difference, according to the abstract, is the explicit spatial-consistency awareness in choosing which tokens to merge. The authors argue this helps maintain segmentation quality while improving efficiency.
The provided abstract is cut off before presenting experimental results, so the source does not include benchmark numbers or comparisons. The contribution described is the method itself and its motivation.