Enhancing the robustness of vision algorithms in real-world scenarios is
challenging. One reason is that existing robustness benchmarks are limited, as
they either rely on synthetic data or ignore the effects of individual nuisance
factors. We introduce OOD-CV, a benchmark dataset that includes
out-of-distribution examples of 10 object categories in terms of pose, shape,
texture, context and the weather conditions, and enables benchmarking models
for image classification, object detection, and 3D pose estimation. In addition
to this novel dataset, we contribute extensive experiments using popular
baseline methods, which reveal that: 1. Some nuisance factors have a much
stronger negative effect on the performance compared to others, also depending
on the vision task. 2. Current approaches to enhance robustness have only
marginal effects, and can even reduce robustness. 3. We do not observe
significant differences between convolutional and transformer architectures. We
believe our dataset provides a rich testbed to study robustness and will help
push forward research in this area.

本文引入 OOD-CV 数据集，并发现某些干扰因素有更强烈的负面影响；当前的提高鲁棒性的方法只有微弱的效果，甚至可能削弱鲁棒性；我们的数据集提供了一个丰富的测试床，以研究鲁棒性，并有助于推进这一领域的研究。