Yes, and no. We ask whether recent progress on the ImageNet classification benchmark continues to represent meaningful generalization, or whether the community has started to overfit to the idiosyncrasies of its labeling procedure. We therefore develop a significantly more robust procedure for collecting human annotations of the ImageNet validation set. Using these new labels, we reassess the accuracy of recently proposed ImageNet classifiers, and find their gains to be substantially smaller than those reported on the original labels. Furthermore, we find the original ImageNet labels to no longer be the best predictors of this independently-collected set, indicating that their usefulness in evaluating vision models may be nearing an end. Nevertheless, we find our annotation procedure to have largely remedied the errors in the original labels, reinforcing ImageNet as a powerful benchmark for future research in visual recognition.

通过重新标注ImageNet数据集的验证集，本文发现现有的ImageNet分类器的性能提升要小于之前的报道，同时发现原始ImageNet标签不再是独立收集集的最佳预测变量，预示其在评估视觉模型方面的用途可能即将结束，但是本文采用的注释程序大大弥补了原始标签中的错误，为未来视觉识别研究提供了重要的基准。

ImageNet 任务是否已完成？