Diffusion models can produce photorealistic inpaintings, but they typically require many sampling steps, which limits practical use. Few-step text-to-image models are faster, yet applying them directly to inpainting often yields visible artifacts and poor blending between the background and the inpainted region. The authors of InverFill trace this problem to the random Gaussian noise used to initialize the process, which under few-step settings loses semantic alignment with the masked image.

InverFill addresses this with a one-step inversion that encodes information from the masked input image into the initial noise. This semantically aligned noise is then fed into a blended sampling pipeline using an off-the-shelf few-step text-to-image model. The method requires no training of a dedicated inpainting model and no real-image supervision, adding only minimal inference overhead.

In experiments, InverFill consistently improved image quality and text coherence across baseline few-step models. At low numbers of function evaluations, it matched the performance of specialized inpainting models while remaining far cheaper. The paper has been accepted to CVPR 2026 as a main conference contribution.