We consider training decision trees using noisily labeled data, focusing on loss functions that can lead to robust learning algorithms. Our contributions are threefold. First, we offer novel theoretical insights on the robustness of many existing loss functions in the context of decision tree learning. We show that some of the losses belong to a class of what we call conservative losses, and the conservative losses lead to an early stopping behavior during training and noise-tolerant predictions during testing. Second, we introduce a framework for constructing robust loss functions, called distribution losses. These losses apply percentile-based penalties based on an assumed margin distribution, and they naturally allow adapting to different noise rates via a robustness parameter. In particular, we introduce a new loss called the negative exponential loss, which leads to an efficient greedy impurity-reduction learning algorithm. Lastly, our experiments on multiple datasets and noise settings validate our theoretical insight and the effectiveness of our adaptive negative exponential loss.

使用含噪标记数据训练决策树，研究能够导致健壮学习算法的损失函数。首先，我们在决策树学习领域提供了有关许多现有损失函数健壮性的新理论见解。其次，我们介绍了一种构建健壮损失函数的框架，称为分布损失。最后，我们的多个数据集和噪声设置上的实验证实了我们的理论洞察力和自适应负指数损失的有效性。

训练带有噪声标签的决策树的鲁棒损失函数