Identifying and handling label errors can significantly enhance the accuracy
of supervised machine learning models. Recent approaches for identifying label
errors demonstrate that a low self-confidence of models with respect to a
certain label represents a good indicator of an erroneous label. However,
latest work has built on softmax probabilities to measure self-confidence. In
this paper, we argue that -- as softmax probabilities do not reflect a model's
predictive uncertainty accurately -- label error detection requires more
sophisticated measures of model uncertainty. Therefore, we develop a range of
novel, model-agnostic algorithms for Uncertainty Quantification-Based Label
Error Detection (UQ-LED), which combine the techniques of confident learning
(CL), Monte Carlo Dropout (MCD), model uncertainty measures (e.g., entropy),
and ensemble learning to enhance label error detection. We comprehensively
evaluate our algorithms on four image classification benchmark datasets in two
stages. In the first stage, we demonstrate that our UQ-LED algorithms
outperform state-of-the-art confident learning in identifying label errors. In
the second stage, we show that removing all identified errors from the training
data based on our approach results in higher accuracies than training on all
available labeled data. Importantly, besides our contributions to the detection
of label errors, we particularly propose a novel approach to generate
realistic, class-dependent label errors synthetically. Overall, our study
demonstrates that selectively cleaning datasets with UQ-LED algorithms leads to
more accurate classifications than using larger, noisier datasets.

通过使用基于不确定性量化的线标错误检测算法（UQ-LED），本研究展示了选择性清理数据集比使用更大、更嘈杂的数据集能够实现更准确的分类。

利用不确定性量化改进标签错误检测与消除

Improving Label Error Detection and Elimination with Uncertainty  Quantification

While data selection methods have been studied extensively in active
learning, data pruning, and data augmentation settings, there is little
evidence for the efficacy of these methods in industry scale settings,
particularly in low-resource languages. Our work presents ways of assessing
prospective training examples in those settings for their "usefulness" or
"difficulty". We also demonstrate how these measures can be used in selecting
important examples for training supervised machine learning models. We
primarily experiment with entropy and Error L2-Norm (EL2N) scores. We use these
metrics to curate high quality datasets from a large pool of \textit{Weak
Signal Labeled} data, which assigns no-defect high confidence hypotheses during
inference as ground truth labels. We then conduct training data augmentation
experiments using these de-identified datasets and demonstrate that score-based
selection can result in a 2% decrease in semantic error rate and 4%-7% decrease
in domain classification error rate when compared to the baseline technique of
random selection.

在工业规模的环境中，特别是在资源有限的语言中，我们通过熵和 EL2N 评分来评估潜在的训练样本的 “有用性” 或 “难度”，并展示了如何使用这些度量来选择用于训练监督机器学习模型的重要样本。我们使用这些度量从大量的 “弱信号标记” 数据中筛选高质量的数据集，然后进行以评分为基础的选择来增强训练数据实验，与随机选择的基线技术相比，结果显示了语义错误率下降 2% 和领域分类错误率下降 4%-7%。

数据选择的熵和边缘得分指标的全面基准评估

Comprehensive Benchmarking of Entropy and Margin Based Scoring Metrics  for Data Selection

Accurately identifying white matter tracts in medical images is essential for
various applications, including surgery planning and tract-specific analysis.
Supervised machine learning models have reached state-of-the-art solving this
task automatically. However, these models are primarily trained on healthy
subjects and struggle with strong anatomical aberrations, e.g. caused by brain
tumors. This limitation makes them unsuitable for tasks such as preoperative
planning, wherefore time-consuming and challenging manual delineation of the
target tract is typically employed. We propose semi-automatic entropy-based
active learning for quick and intuitive segmentation of white matter tracts
from whole-brain tractography consisting of millions of streamlines. The method
is evaluated on 21 openly available healthy subjects from the Human Connectome
Project and an internal dataset of ten neurosurgical cases. With only a few
annotations, the proposed approach enables segmenting tracts on tumor cases
comparable to healthy subjects (dice=0.71), while the performance of automatic
methods, like TractSeg dropped substantially (dice=0.34) in comparison to
healthy subjects. The method is implemented as a prototype named atTRACTive in
the freely available software MITK Diffusion. Manual experiments on tumor data
showed higher efficiency due to lower segmentation times compared to
traditional ROI-based segmentation.

提出一种半自动熵基主动学习方法，通过很少的标注样本，快速直观地从数百万条流线的全脑道路图中分割出白质束，能够在肿瘤病例中实现与正常状况相当的分割效果（dice=0.71），而平均分割时间也比传统基于感兴趣区域的方法更短。