Bayesian neural networks (BNNs) provide calibrated uncertainty estimates, which are valuable in safety-critical applications, but they typically require sampling from weight distributions and running many forward passes. That computational burden has kept them off embedded devices. This paper proposes the Probabilistic Forward Pass (PFP), which assumes Gaussian-distributed weights and activations so that uncertainty can be propagated analytically in a single deterministic forward pass.

The authors built an end-to-end pipeline that trains PFP-based BNNs, compiles them with the TVM deep learning compiler, and deploys them on ARM CPUs. They implemented a dedicated library of Gaussian-propagating operators for multilayer perceptrons and convolutional neural networks, then applied manual and automated tuning. Ablation studies show PFP consistently outperforms stochastic variational inference (SVI) in efficiency, with speedups up to 4200x for small mini-batches.

On the Dirty-MNIST benchmark, PFP-BNNs matched SVI-BNNs in accuracy, uncertainty estimation, and out-of-domain detection while greatly reducing compute cost. The results suggest that combining Bayesian approximations with code generation can make uncertainty-aware models practical on resource-constrained systems. The paper has been accepted for publication in ACM Transactions on Architecture and Code Optimization.