Gradient Boosting Machines (GBM) are hugely popular for solving tabular data
problems. However, practitioners are not only interested in point predictions,
but also in probabilistic predictions in order to quantify the uncertainty of
the predictions. Creating such probabilistic predictions is difficult with
existing GBM-based solutions: they either require training multiple models or
they become too computationally expensive to be useful for large-scale
settings. We propose Probabilistic Gradient Boosting Machines (PGBM), a method
to create probabilistic predictions with a single ensemble of decision trees in
a computationally efficient manner. PGBM approximates the leaf weights in a
decision tree as a random variable, and approximates the mean and variance of
each sample in a dataset via stochastic tree ensemble update equations. These
learned moments allow us to subsequently sample from a specified distribution
after training. We empirically demonstrate the advantages of PGBM compared to
existing state-of-the-art methods: (i) PGBM enables probabilistic estimates
without compromising on point performance in a single model, (ii) PGBM learns
probabilistic estimates via a single model only (and without requiring
multi-parameter boosting), and thereby offers a speedup of up to several orders
of magnitude over existing state-of-the-art methods on large datasets, and
(iii) PGBM achieves accurate probabilistic estimates in tasks with complex
differentiable loss functions, such as hierarchical time series problems, where
we observed up to 10% improvement in point forecasting performance and up to
300% improvement in probabilistic forecasting performance.

提出了一种名为 PGBM 的新方法，利用决策树的随机叶权重和随机树集更新方程逼近训练集中每个样本的均值和方差，从而实现单一模型预测概率分布，比传统方法速度快且效果更佳。