Decision tree ensembles are widely used and competitive learning models.
Despite their success, popular toolkits for learning tree ensembles have
limited modeling capabilities. For instance, these toolkits support a limited
number of loss functions and are restricted to single task learning. We propose
a flexible framework for learning tree ensembles, which goes beyond existing
toolkits to support arbitrary loss functions, missing responses, and multi-task
learning. Our framework builds on differentiable (a.k.a. soft) tree ensembles,
which can be trained using first-order methods. However, unlike classical
trees, differentiable trees are difficult to scale. We therefore propose a
novel tensor-based formulation of differentiable trees that allows for
efficient vectorization on GPUs. We perform experiments on a collection of 28
real open-source and proprietary datasets, which demonstrate that our framework
can lead to 100x more compact and 23% more expressive tree ensembles than those
by popular toolkits.

我们提出了一种灵活的决策树集成学习框架，支持任意损失函数、缺失响应和多任务学习，采用可微分的树集成训练，并通过张量形式的不同 iable trees 实现了高效的向量化，实验结果表明，我们的框架比流行工具包产生的决策树集成更紧凑，表达力更强。