Out-of-distribution (OOD) generalization is a favorable yet challenging
property for deep neural networks. The core challenges lie in the limited
availability of source domains that help models learn an invariant
representation from the spurious features. Various domain augmentation have
been proposed but largely rely on interpolating existing domains and frequently
face difficulties in creating truly "novel" domains. Humans, on the other hand,
can easily extrapolate novel domains, thus, an intriguing question arises: How
can neural networks extrapolate like humans and achieve OOD generalization?
We introduce a novel approach to domain extrapolation that leverages
reasoning ability and the extensive knowledge encapsulated within large
language models (LLMs) to synthesize entirely new domains. Starting with the
class of interest, we query the LLMs to extract relevant knowledge for these
novel domains. We then bridge the gap between the text-centric knowledge
derived from LLMs and the pixel input space of the model using text-to-image
generation techniques. By augmenting the training set of domain generalization
datasets with high-fidelity, photo-realistic images of these new domains, we
achieve significant improvements over all existing methods, as demonstrated in
both single and multi-domain generalization across various benchmarks.
With the ability to extrapolate any domains for any class, our method has the
potential to learn a generalized model for any task without any data. To
illustrate, we put forth a much more difficult setting termed, data-free domain
generalization, that aims to learn a generalized model in the absence of any
collected data. Our empirical findings support the above argument and our
methods exhibit commendable performance in this setting, even surpassing the
supervised setting by approximately 1-2\% on datasets such as VLCS.

利用大型语言模型和文本到图像生成技术，我们介绍了一种领域外推方法，通过合成全新领域的高保真、逼真的图像来显著提高域泛化数据集的训练效果，实现了单一和多领域泛化任务上的显著改进，甚至在没有任何收集数据的条件下，超过了监督设置的表现。