Unsupervised domain adaptation (UDA) adapts a model from a labeled source
domain to an unlabeled target domain in a one-off way. Though widely applied,
UDA faces a great challenge whenever the distribution shift between the source
and the target is large. Gradual domain adaptation (GDA) mitigates this
limitation by using intermediate domains to gradually adapt from the source to
the target domain. In this work, we first theoretically analyze gradual
self-training, a popular GDA algorithm, and provide a significantly improved
generalization bound compared with Kumar et al. (2020). Our theoretical
analysis leads to an interesting insight: to minimize the generalization error
on the target domain, the sequence of intermediate domains should be placed
uniformly along the Wasserstein geodesic between the source and target domains.
The insight is particularly useful under the situation where intermediate
domains are missing or scarce, which is often the case in real-world
applications. Based on the insight, we propose $\textbf{G}$enerative Gradual
D$\textbf{O}$main $\textbf{A}$daptation with Optimal $\textbf{T}$ransport
(GOAT), an algorithmic framework that can generate intermediate domains in a
data-dependent way. More concretely, we first generate intermediate domains
along the Wasserstein geodesic between two given consecutive domains in a
feature space, then apply gradual self-training to adapt the source-trained
classifier to the target along the sequence of intermediate domains.
Empirically, we demonstrate that our GOAT framework can improve the performance
of standard GDA when the given intermediate domains are scarce, significantly
broadening the real-world application scenarios of GDA. Our code is available
at this https URL

我们提出了一种算法框架 GOAT，可在数据相关的方式下生成中间领域，以改进标准的逐渐领域自适应（GDA）的性能，显著扩大 GDA 的实际应用场景。

渐进领域适应：理论与算法

Gradual Domain Adaptation: Theory and Algorithms

The effectiveness of unsupervised domain adaptation degrades when there is a
large discrepancy between the source and target domains. Gradual domain
adaptation (GDA) is one promising way to mitigate such an issue, by leveraging
additional unlabeled data that gradually shift from the source to the target.
Through sequentially adapting the model along the "indexed" intermediate
domains, GDA substantially improves the overall adaptation performance. In
practice, however, the extra unlabeled data may not be separated into
intermediate domains and indexed properly, limiting the applicability of GDA.
In this paper, we investigate how to discover the sequence of intermediate
domains when it is not already available. Concretely, we propose a
coarse-to-fine framework, which starts with a coarse domain discovery step via
progressive domain discriminator training. This coarse domain sequence then
undergoes a fine indexing step via a novel cycle-consistency loss, which
encourages the next intermediate domain to preserve sufficient discriminative
knowledge of the current intermediate domain. The resulting domain sequence can
then be used by a GDA algorithm. On benchmark data sets of GDA, we show that
our approach, which we name Intermediate DOmain Labeler (IDOL), can lead to
comparable or even better adaptation performance compared to the pre-defined
domain sequence, making GDA more applicable and robust to the quality of domain
sequences. Codes are available at this https URL.

本研究提出了一种新的逐步领域自适应方法，称为 Intermediate DOmain Labeler（IDOL），应用了粗到细的框架，通过逐步领域鉴别器训练和循环一致性损失函数，发现中间的领域序列，并提升了自适应性能。

无需索引中间域的渐进式领域自适应

Gradual Domain Adaptation without Indexed Intermediate Domains

The vast majority of existing algorithms for unsupervised domain adaptation
(UDA) focus on adapting from a labeled source domain to an unlabeled target
domain directly in a one-off way. Gradual domain adaptation (GDA), on the other
hand, assumes a path of $(T-1)$ unlabeled intermediate domains bridging the
source and target, and aims to provide better generalization in the target
domain by leveraging the intermediate ones. Under certain assumptions, Kumar et
al. (2020) proposed a simple algorithm, Gradual Self-Training, along with a
generalization bound in the order of $e^{O(T)}
\left(\varepsilon_0+O\left(\sqrt{log(T)/n}\right)\right)$ for the target domain
error, where $\varepsilon_0$ is the source domain error and $n$ is the data
size of each domain. Due to the exponential factor, this upper bound becomes
vacuous when $T$ is only moderately large. In this work, we analyze gradual
self-training under more general and relaxed assumptions, and prove a
significantly improved generalization bound as $\varepsilon_0+ O \left(T\Delta
+ T/\sqrt{n}\right) + \widetilde{O}\left(1/\sqrt{nT}\right)$, where $\Delta$ is
the average distributional distance between consecutive domains. Compared with
the existing bound with an exponential dependency on $T$ as a multiplicative
factor, our bound only depends on $T$ linearly and additively. Perhaps more
interestingly, our result implies the existence of an optimal choice of $T$
that minimizes the generalization error, and it also naturally suggests an
optimal way to construct the path of intermediate domains so as to minimize the
accumulative path length $T\Delta$ between the source and target. To
corroborate the implications of our theory, we examine gradual self-training on
multiple semi-synthetic and real datasets, which confirms our findings. We
believe our insights provide a path forward toward the design of future GDA
algorithms.

本文提出 Gradual Self-Training 算法，通过逐步适应介于源域和目标域之间的中间域，使得在目标域上的泛化误差得到大幅度改善，在更为宽松的假设条件下给出了推导的新的上界，该上界只在 T 上线性地，加性地依赖于一系列基础参数，其中包括平均分布距离，不仅意味着最优的中间域数量存在，还暗示了一种最优的构建方式，同时在多个数据集上验证了该理论可行性。

渐进式领域适应的理解：改进分析、最优路径及其它

Understanding Gradual Domain Adaptation: Improved Analysis, Optimal Path  and Beyond

Machine learning systems must adapt to data distributions that evolve over
time, in applications ranging from sensor networks and self-driving car
perception modules to brain-machine interfaces. We consider gradual domain
adaptation, where the goal is to adapt an initial classifier trained on a
source domain given only unlabeled data that shifts gradually in distribution
towards a target domain. We prove the first non-vacuous upper bound on the
error of self-training with gradual shifts, under settings where directly
adapting to the target domain can result in unbounded error. The theoretical
analysis leads to algorithmic insights, highlighting that regularization and
label sharpening are essential even when we have infinite data, and suggesting
that self-training works particularly well for shifts with small
Wasserstein-infinity distance. Leveraging the gradual shift structure leads to
higher accuracies on a rotating MNIST dataset and a realistic Portraits
dataset.

论文研究了机器学习系统对随时间变化而演化的数据分布的适应性，在渐进领域适应的情况下，证明了使用自我训练的非空界，通过正则化和标签增强来提高算法精度，特别是对于 Wasserstein-infinity 距离较小的转化数据和真实肖像数据集。