Major advancements in computer vision can primarily be attributed to the use of labeled datasets. However, acquiring labels for datasets often results in errors which can harm model performance. Recent works have proposed methods to automatically identify mislabeled images, but developing strategies to effectively implement them in real world datasets has been sparsely explored. Towards improved data-centric methods for cleaning real world vision datasets, we first conduct more than 200 experiments carefully benchmarking recently developed automated mislabel detection methods on multiple datasets under a variety of synthetic and real noise settings with varying noise levels. We compare these methods to a Simple and Efficient Mislabel Detector (SEMD) that we craft, and find that SEMD performs similarly to or outperforms prior mislabel detection approaches. We then apply SEMD to multiple real world computer vision datasets and test how dataset size, mislabel removal strategy, and mislabel removal amount further affect model performance after retraining on the cleaned data. With careful design of the approach, we find that mislabel removal leads per-class performance improvements of up to 8% of a retrained classifier in smaller data regimes.

计算机视觉中使用标记数据集是主要的进展，近期的研究工作提出了用于自动识别错误标记图像的方法，但在实际数据集中有效实施这些方法的策略仍未充分探索。本研究在多个数据集上对最近发展的自动化错误标记检测方法进行了详细评估，结果表明该方法在清理真实世界视觉数据集中性能相似甚至优于先前方法。通过精心设计方法，将该方法应用于真实世界计算机视觉数据集后，在较小的数据情况下使每个分类器的性能提高了8%。

现实世界视觉数据集中自动误标签检测的实证研究