Diffusion transformers (DiTs) are now a common backbone for high-quality text-to-image and text-to-video generation, but they face a heavy inference bottleneck: dense self-attention must run repeatedly over long token sequences across many denoising steps. The cost grows as the token sequence lengthens, making efficient inference an important practical problem.
The paper proposes Sink-Sparse Hybrid Attention, a train-free replacement for standard attention that avoids recomputing full dense attention at every step. The authors show that pretrained DiTs contain reusable, layer- and head-specific high-mass structure in their attention maps, which can be identified from a small offline calibration set and carried over to future generations. The method reuses static sink masks from that calibration and adds a lightweight pooled-KV branch; model weights and sampling schedules remain unchanged.
On the video model Wan2.1-T2V-1.3B, the authors report a 1.54x end-to-end speedup while maintaining generation quality. On the image model Z-Image-Turbo, they achieve a 1.57x end-to-end speedup with competitive GenEval2 performance. These results point toward reusable importance-aware sparse support as an effective, training-free acceleration path for both T2V and T2I diffusion transformers.