Domain Generalization (DG) aims to learn models whose performance remains
high on unseen domains encountered at test-time by using data from multiple
related source domains. Many existing DG algorithms reduce the divergence
between source distributions in a representation space to potentially align the
unseen domain close to the sources. This is motivated by the analysis that
explains generalization to unseen domains using distributional distance (such
as the Wasserstein distance) to the sources. However, due to the openness of
the DG objective, it is challenging to evaluate DG algorithms comprehensively
using a few benchmark datasets. In particular, we demonstrate that the accuracy
of the models trained with DG methods varies significantly across unseen
domains, generated from popular benchmark datasets. This highlights that the
performance of DG methods on a few benchmark datasets may not be representative
of their performance on unseen domains in the wild. To overcome this roadblock,
we propose a universal certification framework based on distributionally robust
optimization (DRO) that can efficiently certify the worst-case performance of
any DG method. This enables a data-independent evaluation of a DG method
complementary to the empirical evaluations on benchmark datasets. Furthermore,
we propose a training algorithm that can be used with any DG method to provably
improve their certified performance. Our empirical evaluation demonstrates the
effectiveness of our method at significantly improving the worst-case loss
(i.e., reducing the risk of failure of these models in the wild) without
incurring a significant performance drop on benchmark datasets.

本文提出了一种基于分布鲁棒优化的普适认证框架，旨在弥补现有基准数据集在测试时无法全面评估领域通用算法的局限性，并提出了一种训练算法，可以用于改进其认证性能。实证评估表明，该方法显著提高了风险压力下模型的最坏损失，而在基准数据集上并未出现显著的性能下降。

关于认证和提高对未见领域的泛化能力

On Certifying and Improving Generalization to Unseen Domains

Image hashing is a principled approximate nearest neighbor approach to find
similar items to a query in a large collection of images. Hashing aims to learn
a binary-output function that maps an image to a binary vector. For optimal
retrieval performance, producing balanced hash codes with low-quantization
error to bridge the gap between the learning stage's continuous relaxation and
the inference stage's discrete quantization is important. However, in the
existing deep supervised hashing methods, coding balance and low-quantization
error are difficult to achieve and involve several losses. We argue that this
is because the existing quantization approaches in these methods are
heuristically constructed and not effective to achieve these objectives. This
paper considers an alternative approach to learning the quantization
constraints. The task of learning balanced codes with low quantization error is
re-formulated as matching the learned distribution of the continuous codes to a
pre-defined discrete, uniform distribution. This is equivalent to minimizing
the distance between two distributions. We then propose a computationally
efficient distributional distance by leveraging the discrete property of the
hash functions. This distributional distance is a valid distance and enjoys
lower time and sample complexities. The proposed single-loss quantization
objective can be integrated into any existing supervised hashing method to
improve code balance and quantization error. Experiments confirm that the
proposed approach substantially improves the performance of several
representative hashing~methods.

本文提出了一种替代学习量化限制的方法，将学习平衡码并降低量化误差的任务重新定义为将连续码的学习分布与预定义的离散均匀分布匹配，从而最小化两个分布之间的距离。同时，本文还提出了一种计算有效的分布距离，将其集成到现有的监督式哈希方法中能够提高代码平衡性和量化误差的性能表现。

量化的一种丧失：离散 Wasserstein 分布匹配的深层哈希

One Loss for Quantization: Deep Hashing with Discrete Wasserstein  Distributional Matching

The vast majority of existing algorithms for unsupervised domain adaptation
(UDA) focus on adapting from a labeled source domain to an unlabeled target
domain directly in a one-off way. Gradual domain adaptation (GDA), on the other
hand, assumes a path of $(T-1)$ unlabeled intermediate domains bridging the
source and target, and aims to provide better generalization in the target
domain by leveraging the intermediate ones. Under certain assumptions, Kumar et
al. (2020) proposed a simple algorithm, Gradual Self-Training, along with a
generalization bound in the order of $e^{O(T)}
\left(\varepsilon_0+O\left(\sqrt{log(T)/n}\right)\right)$ for the target domain
error, where $\varepsilon_0$ is the source domain error and $n$ is the data
size of each domain. Due to the exponential factor, this upper bound becomes
vacuous when $T$ is only moderately large. In this work, we analyze gradual
self-training under more general and relaxed assumptions, and prove a
significantly improved generalization bound as $\varepsilon_0+ O \left(T\Delta
+ T/\sqrt{n}\right) + \widetilde{O}\left(1/\sqrt{nT}\right)$, where $\Delta$ is
the average distributional distance between consecutive domains. Compared with
the existing bound with an exponential dependency on $T$ as a multiplicative
factor, our bound only depends on $T$ linearly and additively. Perhaps more
interestingly, our result implies the existence of an optimal choice of $T$
that minimizes the generalization error, and it also naturally suggests an
optimal way to construct the path of intermediate domains so as to minimize the
accumulative path length $T\Delta$ between the source and target. To
corroborate the implications of our theory, we examine gradual self-training on
multiple semi-synthetic and real datasets, which confirms our findings. We
believe our insights provide a path forward toward the design of future GDA
algorithms.

本文提出 Gradual Self-Training 算法，通过逐步适应介于源域和目标域之间的中间域，使得在目标域上的泛化误差得到大幅度改善，在更为宽松的假设条件下给出了推导的新的上界，该上界只在 T 上线性地，加性地依赖于一系列基础参数，其中包括平均分布距离，不仅意味着最优的中间域数量存在，还暗示了一种最优的构建方式，同时在多个数据集上验证了该理论可行性。