Sparse variational Gaussian processes (SVGPs) are a standard tool for scalable Gaussian process modeling because they maintain a principled lower bound on the full model's marginal likelihood. Recent work has extended SVGPs by modifying the posterior conditional distribution, for instance with a block-diagonal structured scaling matrix that produces a tighter lower bound. A new arXiv preprint by Matteo Hoxha asks whether selecting blocks specifically to tighten that variational objective also improves test RMSE and negative log-likelihood at a fixed inducing-point budget.
The paper's key contribution is a simple, geometry-aware algorithm for choosing a partition of the data that targets the scaling matrix's contribution to the bound, which enters through an upper bound on the log-determinant of the residual covariance matrix. The author reports explicit computational complexity and low overhead, and shows that the algorithm recovers a large fraction of this log-determinant contribution. Training is handled with a doubly stochastic optimization framework, keeping the method scalable for large datasets.
The preprint also explores whether sparse Gaussian process methods based on alternative conditional posterior structures can improve learning of hyperparameters and inducing points for SVGP. Since this digest draws on a single arXiv source, there are no independent results to compare; the findings are best treated as preliminary until peer review and replication.