Bayesian neural networks offer a principled way to quantify epistemic uncertainty, but their usual weight-space priors produce high-dimensional posteriors that are computationally prohibitive to sample from. In reinforcement learning, the paper argues, approximate inference schemes often fail to capture this uncertainty, a problem that is especially acute for exploration.
The paper introduces DP-BNNs, a new class of Bayesian neural networks that place Dirichlet process priors on the underlying data-generating distribution instead of on network parameters. On top of this, it builds DP-DQN, an RL algorithm that maintains posterior distributions over value functions and performs efficient Thompson sampling. The method combines data-driven posterior reweighting via the Bayesian bootstrap with generative prior hypotheses injected through the Dirichlet process base measure, enabling deep exploration with a single neural network and avoiding the overhead of multi-network ensembles.
Published in Transactions on Machine Learning Research, the work is a single-author study; as the only source, its claims are presented without comparison to other work.