Tree ensembles such as Random Forests have achieved impressive empirical
success across a wide variety of applications. To understand how these models
make predictions, people routinely turn to feature importance measures
calculated from tree ensembles. It has long been known that Mean Decrease
Impurity (MDI), one of the most widely used measures of feature importance,
incorrectly assigns high importance to noisy features, leading to systematic
bias in feature selection. In this paper, we address the feature selection bias
of MDI from both theoretical and methodological perspectives. Based on the
original definition of MDI by Breiman et al. for a single tree, we derive a
tight non-asymptotic bound on the expected bias of MDI importance of noisy
features, showing that deep trees have higher (expected) feature selection bias
than shallow ones. However, it is not clear how to reduce the bias of MDI using
its existing analytical expression. We derive a new analytical expression for
MDI, and based on this new expression, we are able to propose a debiased MDI
feature importance measure using out-of-bag samples, called MDI-oob. For both
the simulated data and a genomic ChIP dataset, MDI-oob achieves
state-of-the-art performance in feature selection from Random Forests for both
deep and shallow trees.

该论文从理论和方法论两方面，通过提出一种基于无袋预测样本的去偏 MDI 重要性度量方法，即 MDI-oob，来解决 MDI（Mean Decrease Impurity）特征选择方法中的偏差问题，并在其基础上表现出 RF（Random Forests）特征选择方面的最佳性能。