Training deep neural networks with binary or ternary weights is attractive for edge devices because it drastically reduces memory and compute demands. However, the discrete nature of the weights makes gradient-based training difficult. A new arXiv preprint offers a different angle: it shows that the training loss for such networks can be expressed as a multilinear polynomial over the discrete weights, which is structurally identical to the energy of a hierarchical high-order Ising model.
This equivalence lets the authors borrow tools from statistical physics. They introduce a training procedure based on noise-shaped sampling and annealing, which navigates the discrete weight space in a way that mirrors how physical systems settle into low-energy states. The goal is to make training itself feasible directly on edge hardware, rather than only running inference there.
The abstract does not include experimental results or benchmarks, so the practical gains remain to be seen. But the theoretical framing is notable: it connects a practical machine-learning problem to a well-studied class of models in physics, potentially opening the door to new algorithms for on-device learning.