Diffusion models represent concepts in ways that shift as information passes through network layers and across denoising steps. This drift makes it difficult to track a feature consistently, but current interpretability tools have not kept up: sparse autoencoders (SAEs) typically build separate dictionaries for each block or each timestep, so no single dictionary spans the whole process.

A new paper on arXiv introduces SharedSAE, a unified sparse dictionary designed to work across U-Net blocks and denoising times. The motivation is straightforward: without a shared dictionary, there is no common basis for localizing features throughout the model's forward pass. SharedSAE aims to fill that gap by offering one set of features that remains valid across depth and time.

The abstract does not report experimental results, so the claims are limited to the proposal itself. Still, the direction is clear: instead of stitching together many per-layer or per-step dictionaries, a single shared dictionary could make diffusion model features easier to interpret and compare. Whether SharedSAE delivers on that promise will depend on the full paper's evaluation.