Learned inference systems often use priors to reconstruct regions that measurements barely reach and to report uncertainty there. But they can fail silently: the reported interval excludes the true value while every ground-truth-free diagnostic stays within its noise. The new arXiv preprint calls these errors the blind set and asks whether they can be predicted before the target data are seen.
For a correctly specified linear-Gaussian model, the risk of a silent failure factorises exactly into non-detection and non-coverage, but this does not hold for learned systems in general. The author builds a score that approximates this reference by linearising a VAE prior. In a prospective study on real X-ray CT objects under simulated restricted access, with predictions committed by hash before any trial, the score ranked the realised blindness of two learned inference systems—latent Laplace and ensemble Kalman inversion—with Spearman rho 0.79 and 0.84 over 240 pre-specified errors. Simple structural predictors would also have passed these tests.
Sequential post-hoc analyses, each specified before its own computation, showed that under bounded reach the learned score gave no detectable improvement over a physics-based baseline in rankings or trial-level log scores. At the weakest attenuation studied, prior information improved rankings over the physics-based baseline, most clearly for ensemble Kalman inversion, but a Gaussian prior did as well and the gain was not detectable at the trial level. The author notes the score is not a calibrated probability.
The main conclusion is that silent failures can be anticipated in this setting, but their structure follows mainly from measurement access and the decision quantity rather than from the learned prior.