Enhancing the robustness of vision algorithms in real-world scenarios is challenging. One reason is that existing robustness benchmarks are limited, as they either rely on synthetic data or ignore the effects of individual nuisance factors. We introduce OOD-CV-v2, a benchmark dataset that includes out-of-distribution examples of 10 object categories in terms of pose, shape, texture, context and the weather conditions, and enables benchmarking of models for image classification, object detection, and 3D pose estimation. In addition to this novel dataset, we contribute extensive experiments using popular baseline methods, which reveal that: 1) Some nuisance factors have a much stronger negative effect on the performance compared to others, also depending on the vision task. 2) Current approaches to enhance robustness have only marginal effects, and can even reduce robustness. 3) We do not observe significant differences between convolutional and transformer architectures. We believe our dataset provides a rich test bed to study robustness and will help push forward research in this area. Our dataset can be accessed from http://www.ood-cv.org/challenge.html

我们引入了OOD-CV-v2数据集，其中包括了10个物体类别的姿态、形状、纹理、上下文和天气条件之外的分布数据，以提高图像分类、物体检测和3D姿态估计模型的鲁棒性性能。经过大量的实验，我们发现当前的提高鲁棒性方法只有微小的作用，并且可能会降低鲁棒性；同时，在卷积和transformer架构之间没有明显的差异。

OOD-CV-v2：自然图像中个体模糊的各类变化鲁棒性的扩展基准