Large-scale datasets for single-label multi-class classification, such as
\emph{ImageNet-1k}, have been instrumental in advancing deep learning and
computer vision. However, a critical and often understudied aspect is the
comprehensive quality assessment of these datasets, especially regarding
potential multi-label annotation errors. In this paper, we introduce a
lightweight, user-friendly, and scalable framework that synergizes human and
machine intelligence for efficient dataset validation and quality enhancement.
We term this novel framework \emph{Multilabelfy}. Central to Multilabelfy is an
adaptable web-based platform that systematically guides annotators through the
re-evaluation process, effectively leveraging human-machine interactions to
enhance dataset quality. By using Multilabelfy on the ImageNetV2 dataset, we
found that approximately $47.88\%$ of the images contained at least two labels,
underscoring the need for more rigorous assessments of such influential
datasets. Furthermore, our analysis showed a negative correlation between the
number of potential labels per image and model top-1 accuracy, illuminating a
crucial factor in model evaluation and selection. Our open-source framework,
Multilabelfy, offers a convenient, lightweight solution for dataset
enhancement, emphasizing multi-label proportions. This study tackles major
challenges in dataset integrity and provides key insights into model
performance evaluation. Moreover, it underscores the advantages of integrating
human expertise with machine capabilities to produce more robust models and
trustworthy data development. The source code for Multilabelfy will be
available at this https URL
\keywords{Computer Vision \and Dataset Quality Enhancement \and Dataset
Validation \and Human-Computer Interaction \and Multi-label Annotation.}

本研究引入了一个轻量级、用户友好、可扩展的框架 “Multilabelfy”，通过人机智能的协同作用，实现了高效的数据集验证和质量提升，同时突出了多标签占比的重要性，发现 ImageNetV2 数据集中约 47.88% 的图像包含至少两个标签，这凸显了对这类有影响力的数据集更严格评估的需求，并揭示了潜在标签数与模型准确性之间的负相关关系。