Gradient Boosted Decision Trees (GBDTs) are widely used for building ranking and relevance models in search and recommendation. Considerations such as latency and interpretability dictate the use of as few features as possible to train these models. Feature selection in GBDT models typically involves heuristically ranking the features by importance and selecting the top few, or by performing a full backward feature elimination routine. On-the-fly feature selection methods proposed previously scale suboptimally with the number of features, which can be daunting in high dimensional settings. We develop a scalable forward feature selection variant for GBDT, via a novel group testing procedure that works well in high dimensions, and enjoys favorable theoretical performance and computational guarantees. We show via extensive experiments on both public and proprietary datasets that the proposed method offers significant speedups in training time, while being as competitive as existing GBDT methods in terms of model performance metrics. We also extend the method to the multitask setting, allowing the practitioner to select common features across tasks, as well as selecting task-specific features.

本文提出了一种可伸缩的基于前向特征选择的梯度提升决策树方法，通过一种新颖的群体测试过程，在高维度下工作良好，并具有较好的理论性能和计算保证，可用于搜索和推荐中的排名和相关模型的构建。同时，该方法扩展到多任务设置，允许从多个任务中选择通用特征和选择任务特定特征，与现有的梯度提升决策树方法相比，在模型性能度量方面表现相当，但训练时间有显著提升。

（多任务）梯度提升树的可扩展特征选择