Systems that treat a variational autoencoder's latent space as a set of discrete codes depend on a simple consistency property: a latent should be read back as the same code it was written with. A common way to test this is to check whether reconstruction is perfect. But a new arXiv preprint argues that this test is not sufficient.

The paper's central claim is that perfect reconstruction cannot tell you whether two codebooks on a VAE's latent space actually agree. Two different codebooks could both reconstruct the original data perfectly, yet assign different discrete codes to the same latent. In other words, the reconstruction objective does not pin down a unique codebook.

The authors trace the issue to an assumption in Shannon's 1948 work, which took a shared codebook as a given. Inside a VAE, that assumption is not automatically satisfied. The abstract does not provide a proposed fix, but the result suggests that researchers relying on reconstruction fidelity to validate codebook alignment may need a stronger test.