While several methodologies have been proposed for the daunting task of
domain generalization, understanding what makes this task challenging has
received little attention. Here we present SemanticDG (Semantic Domain
Generalization): a benchmark with 15 photo-realistic domains with the same
geometry, scene layout and camera parameters as the popular 3D ScanNet dataset,
but with controlled domain shifts in lighting, materials, and viewpoints. Using
this benchmark, we investigate the impact of each of these semantic shifts on
generalization independently. Visual recognition models easily generalize to
novel lighting, but struggle with distribution shifts in materials and
viewpoints. Inspired by human vision, we hypothesize that scene context can
serve as a bridge to help models generalize across material and viewpoint
domain shifts and propose a context-aware vision transformer along with a
contrastive loss over material and viewpoint changes to address these domain
shifts. Our approach (dubbed as CDCNet) outperforms existing domain
generalization methods by over an 18% margin. As a critical benchmark, we also
conduct psychophysics experiments and find that humans generalize equally well
across lighting, materials and viewpoints. The benchmark and computational
model introduced here help understand the challenges associated with
generalization across domains and provide initial steps towards extrapolation
to semantic distribution shifts. We include all data and source code in the
supplement.

该研究提出了一种基于语义的跨领域泛化基准（SemanticDG），并使用这个基准探索了在材料、视角等不同领域转移时的影响，通过提出一种上下文感知的视觉转换器，改进了现有领域泛化算法，实现了 18% 以上的性能提升，最终成功解决了领域泛化任务存在的挑战。