In this paper, we present Partially Stochastic Infinitely Deep Bayesian Neural Networks, a novel family of architectures that integrates partial stochasticity into the framework of infinitely deep neural networks. Our new class of architectures is designed to improve the limitations of existing architectures around computational efficiency at training and inference time. To do this, we leverage the advantages of partial stochasticity in the infinite-depth limit which include the benefits of full stochasticity e.g. robustness, uncertainty quantification, and memory efficiency, whilst improving their limitations around computational efficiency at training and inference time. We present a variety of architectural configurations, offering flexibility in network design including different methods for weight partition. We also provide mathematical guarantees on the expressivity of our models by establishing that our network family qualifies as Universal Conditional Distribution Approximators. Lastly, empirical evaluations across multiple tasks show that our proposed architectures achieve better downstream task performance and uncertainty quantification than their counterparts while being significantly more efficient.

本文提出部分随机性无限深度贝叶斯神经网络(Partially Stochastic Infinitely Deep Bayesian Neural Networks)，将部分随机性融入无限深度神经网络框架中，并通过改进现有的架构，提高了训练和推理的计算效率以及稳健性、不确定性量化和内存效率。作者在实验中通过多个任务的经验评估证明了该架构在下游任务性能和不确定性量化方面优于现有架构，并且更加高效。

部分随机无穷深贝叶斯神经网络