Probabilistic circuits (PCs) are generative models that support efficient probabilistic inference, making them attractive for applications where exact queries matter. One common way to increase their expressiveness is to add latent states—hidden variables that can capture unobserved structure in the data.
However, a new paper on arXiv reports diminishing returns from this strategy. Adding latent states does increase a PC's representational capacity, but the resulting likelihood gains become smaller with each additional state. The authors study how latent states are utilized, and their analysis points to a saturation effect.
This trade-off has practical implications. Model designers must weigh the added complexity of more latent states against the shrinking improvements in likelihood. The findings suggest that blindly scaling up latent states may not be the most efficient path to better generative models, and they open the door to more targeted ways of allocating representational capacity.