As synthetic data becomes a common substitute for scarce real data, a key question is when a classifier trained on synthetic inputs will work on the real distribution. Odunlami, Riedlinger, and Gottschalk provide theoretical bounds for binary classification under a shared conditional label distribution, and they identify a sharp failure mode.
When the real covariate distribution is absolutely continuous with respect to the synthetic distribution with a controlled density ratio, they derive finite-sample excess-error bounds. But in the partially or fully singular case—where real data has mass outside the synthetic support—they show that for sufficiently expressive hypothesis classes, synthetic risk alone cannot uniformly control real-world error. A classifier can be nearly optimal on synthetic data while approaching worst-case error on the uncovered real region, even when the two distributions are arbitrarily close in Wasserstein distance.
The authors recover a transfer guarantee by assuming Lipschitz continuity of hypotheses and the log-odds of the conditional label probability; then the real error is bounded by the probability that a real sample lies outside the reach of the synthetic data. Experiments on low-dimensional settings and on 32,200 images generated as counterparts to real ImageNet images confirm the theory: an adversarial classifier achieved 0.5% error on synthetic images but 99.25% error on real images, while standard architectures showed positively correlated synthetic and real test errors. The authors recommend budgeting synthetic data at high sample counts and validating models on real data.