Diffusion models rely on neural networks to estimate score functions, but the architectural biases of these networks limit the class of functions they can represent. A new preprint on arXiv notes that how these biases affect generalization is still poorly understood.
The paper studies denoising score matching using score networks with a rank constraint. The abstract indicates the authors are investigating how this architectural restriction influences the model's ability to generalize beyond its training data.
Because the abstract is truncated, specific results are not available in the source. The study appears to be an early theoretical contribution to understanding the role of rank constraints in diffusion model training.