Researchers have adapted activation steering—a technique previously used for controlling generation in autoregressive models—to diffusion language models (DLMs). Their work, described in a recent arXiv paper, focuses on a new task that arises from the unique inference process of DLMs: modifying an existing text sequence in-place so that it expresses a different concept.

The authors call this "in-place text sequence modification" and propose a system named TimpaTeks to address it. While the abstract does not detail the inner workings, the approach appears to build on activation steering to guide the diffusion process toward a target concept without regenerating the text from scratch.

This is among the first attempts to apply steering to diffusion-based language models, which have different inference dynamics than standard transformers. The authors frame it as a novel problem that could enable more controlled editing of generated or existing text.