Diffusion transformers are a core building block for generating video and images, but their attention layers scale quadratically with sequence length, putting a hard limit on throughput. Sparse attention is a natural remedy: rather than letting every token interact with every other token, restrict each token to a subset of positions and the compute drops substantially.
The new preprint takes aim at how that subset is chosen. Where prior work uses block sparse attention — attending only to certain blocks of positions — the authors propose Piecewise Sparse Attention (PISA), which they argue is "wiser" for efficient diffusion transformers. The title itself frames PISA as an improvement over the blockwise approach, suggesting the shape of the sparse pattern matters, not just the act of pruning.
This appears to be a revised version of the paper, flagged as a "replace" announcement on arXiv. Because the abstract is truncated in the source, specific speedups or quality trade-offs are not available here; what is clear is the motivation: diffusion transformers are too slow for practical generation, and PISA targets the attention bottleneck directly.