Domain adaptation is an important problem and often needed for real-world
applications. In this problem, instead of i.i.d. training and testing
datapoints, we assume that the source (training) data and the target (testing)
data have different distributions. With that setting, the empirical risk
minimization training procedure often does not perform well, since it does not
account for the change in the distribution. A common approach in the domain
adaptation literature is to learn a representation of the input that has the
same (marginal) distribution over the source and the target domain. However,
these approaches often require additional networks and/or optimizing an
adversarial (minimax) objective, which can be very expensive or unstable in
practice. To improve upon these marginal alignment techniques, in this paper,
we first derive a generalization bound for the target loss based on the
training loss and the reverse Kullback-Leibler (KL) divergence between the
source and the target representation distributions. Based on this bound, we
derive an algorithm that minimizes the KL term to obtain a better
generalization to the target domain. We show that with a probabilistic
representation network, the KL term can be estimated efficiently via minibatch
samples without any additional network or a minimax objective. This leads to a
theoretically sound alignment method which is also very efficient and stable in
practice. Experimental results also suggest that our method outperforms other
representation-alignment approaches.

本文研究了领域自适应中的输入表示对齐问题，提出了一种基于最小化源域和目标域表示分布之间的反向 Kullback-Leibler 差距的方法，利用概率表示网络实现了高效稳定的对齐方法，实验结果表明其优于其他对齐方法。

KL 导向领域适应

KL Guided Domain Adaptation

Despite the success of generative adversarial networks (GANs) for image
generation, the trade-off between visual quality and image diversity remains a
significant issue. This paper achieves both aims simultaneously by improving
the stability of training GANs. The key idea of the proposed approach is to
implicitly regularize the discriminator using representative features. Focusing
on the fact that standard GAN minimizes reverse Kullback-Leibler (KL)
divergence, we transfer the representative feature, which is extracted from the
data distribution using a pre-trained autoencoder (AE), to the discriminator of
standard GANs. Because the AE learns to minimize forward KL divergence, our GAN
training with representative features is influenced by both reverse and forward
KL divergence. Consequently, the proposed approach is verified to improve
visual quality and diversity of state of the art GANs using extensive
evaluations.

本论文使用预训练自编码器的代表特征隐式规范鉴别器，通过影响反向和正向 KL 距离使得 GAN 的训练更稳定，在图像质量和多样性上都有提高。