The lack of standardized robustness metrics and the widespread reliance on
numerous unrelated benchmark datasets for testing have created a gap between
academically validated robust models and their often problematic practical
adoption. To address this, we introduce XIMAGENET-12, an explainable benchmark
dataset with over 200K images and 15,600 manual semantic annotations. Covering
12 categories from ImageNet to represent objects commonly encountered in
practical life and simulating six diverse scenarios, including overexposure,
blurring, color changing, etc., we further propose a novel robustness criterion
that extends beyond model generation ability assessment. This benchmark
dataset, along with related code, is available at
this https URL Researchers and practitioners
can leverage this resource to evaluate the robustness of their visual models
under challenging conditions and ultimately benefit from the demands of
practical computer vision systems.

为了解决标准化鲁棒性评估指标缺乏和过多无关基准数据集的问题，我们引入了 XIMAGENET-12，一个可解释的基准数据集，包含 20 万多张图像和 15,600 个手动语义注释。同时，我们提出了一个新的鲁棒性评估标准，超越了模型生成能力的评估。研究人员和从业者可以利用这个资源，在具有挑战性的条件下评估他们的视觉模型的鲁棒性，并从实际的计算机视觉系统需求中受益。