Skip to main content
Back to timeline
arXivSource publication:

Bayesian neural network output distributions get a sound envelope: a nonlinear abstract domain cuts total variation distance by up to about 950x over interval abstraction

Synopsis

Bayesian neural networks treat weights as distributions, yet prior analyses either sampled or propagated only expectations and entropy, leaving the full output distribution without guarantees. This work builds a marginal collecting semantics and represents probability measures with exponential-affine bounds over a partition of the support, yielding composable abstract transformers that give a sound envelope of the complete output distribution on several regression benchmarks. Against an interval abstract domain, total variation distance improves by up to about 950x, at the cost of longer runtimes.

Source-provided article image: A Nonlinear Abstract Domain for Bayesian Neural Networks
arXiv

Interpretation

The complete output distribution of a Bayesian neural network can now be soundly enclosed, rather than approximated from finitely many samples or summarized by expectations, variance and entropy alone. Sampling methods reduce the Bayesian network to a deterministic one before analysis, so their guarantees depend on the drawn samples and rare tail events are easily missed; methods that propagate selected statistics can only answer pre-specified queries. On feed-forward Bayesian networks trained on three UCI regression datasets, compared with an interval abstract domain under the same design, total variation distance on one Energy benchmark falls from 298206.33 to 16858.80, a ratio of 17.69, and on another from 3665.16 to 20.45, a ratio of 179.24.

Each neuron's probability measure is split along its support into intervals, and on each interval exponential-affine functions bound the density from above and below, representing continuous densities and the discrete point masses introduced by ReLU together. Linear representations struggle to capture nonlinear shapes such as Gaussian densities, while truncating unbounded tails discards non-negligible probability mass. For a Gaussian density the logarithm is taken, concavity of the log-density gives a secant lower bound and a tangent upper bound, and exponentiating returns density bounds; unbounded tails get a conservative constant lower bound, and the affine layer rewrites marginalization as a sum of conditional expectations weighted by input-box probabilities.

Perspective

The demonstrated setting is pre-trained feed-forward Bayesian networks with mean-field Gaussian posteriors and ReLU activations on regression tasks, with hidden widths from 4 to 128; generality is checked on simplified Bayesian inference programs. The intended readers are researchers and engineers working on uncertainty quantification and formal reasoning.

Precision decreases as network size grows, and tail probability estimates are tighter on small networks; because constant bounds over unbounded intervals are non-integrable, the interval-domain baseline is compared only under bounded support.