Deep latent variable models are widely used in fields like single-cell genomics, yet their prior distributions receive little attention. The VampPrior approximates the optimal prior—the aggregate posterior—and often achieves high model evidence. The authors identify two key limitations: it ties prior and posterior uncertainty to a single amortized covariance, and its fixed uniform mixture weights lead to poor clustering.
To address these issues, they "unleash" the VampPrior by decoupling the prior and posterior covariances, allowing the model to generate high prior entropy and low posterior entropy without sharing parameters. They also replace the uniform mixture with a Dirichlet process, providing a non-parametric clustering mechanism. Combined, these changes yield higher model evidence than the original VampPrior.
The authors demonstrate practical impact by augmenting scVI, a variational inference tool for single-cell data, with their new prior. Across six atlases, they report improved single-cell integration and clustering, along with significantly higher model evidence. Because the modification only affects the prior, it can be applied to any deep latent variable model with a continuous latent space.