We apply Bayesian Additive Regression Tree (BART) principles to training an
ensemble of small neural networks for regression tasks. Using Markov Chain
Monte Carlo, we sample from the posterior distribution of neural networks that
have a single hidden layer. To create an ensemble of these, we apply Gibbs
sampling to update each network against the residual target value (i.e.
subtracting the effect of the other networks). We demonstrate the effectiveness
of this technique on several benchmark regression problems, comparing it to
equivalent shallow neural networks, BART, and ordinary least squares. Our
Bayesian Additive Regression Networks (BARN) provide more consistent and often
more accurate results. On test data benchmarks, BARN averaged between 5 to 20
percent lower root mean square error. This error performance does come at the
cost, however, of greater computation time. BARN sometimes takes on the order
of a minute where competing methods take a second or less. But, BARN without
cross-validated hyperparameter tuning takes about the same amount of
computation time as tuned other methods. Yet BARN is still typically more
accurate.

应用贝叶斯加法回归树（BART）原理对小型神经网络进行回归任务的训练，通过马尔科夫链蒙特卡洛从具有单隐藏层的神经网络的后验分布中进行采样并应用吉布斯采样来更新每个网络，演示了该技术在多个基准回归问题上的有效性，并与等效的浅层神经网络、BART 和最小二乘法进行比较，我们的贝叶斯加法回归网络（BARN）提供了更一致且通常更准确的结果。在测试数据基准中，BARN 的平均根均方误差较低，降低了 5% 至 20%。然而，这种误差性能的提升代价是更长的计算时间，BARN 的计算时间可能为一分钟，而竞争方法只需一秒或更短，但是没有经过交叉验证的超参数调整的 BARN 与其他方法总体上需要相同的计算时间，然而 BARN 通常更准确。

贝叶斯加性回归网络

Bayesian Additive Regression Networks

In this paper we develop the randomized Sharded Bayesian Additive Regression
Trees (SBT) model. We introduce a randomization auxiliary variable and a
sharding tree to decide partitioning of data, and fit each partition component
to a sub-model using Bayesian Additive Regression Tree (BART). By observing
that the optimal design of a sharding tree can determine optimal sharding for
sub-models on a product space, we introduce an intersection tree structure to
completely specify both the sharding and modeling using only tree structures.
In addition to experiments, we also derive the theoretical optimal weights for
minimizing posterior contractions and prove the worst-case complexity of SBT.

开发了基于随机切片的贝叶斯加性回归树（SBT）模型，通过引入随机化辅助变量和分片树来决定数据分片，并使用贝叶斯加性回归树（BART）将每个分区组件拟合到子模型中。通过引入交集树结构完全指定切割和建模，通过实验和推导理论优化权重来最小化后验收缩并证明了 SBT 的最坏时间复杂性。