Unsupervised domain adaptation (UDA) tries to overcome the need for a large
labeled dataset by transferring knowledge from a source dataset, with lots of
labeled data, to a target dataset, that has no labeled data. Since there are no
labels in the target domain, early misalignment might propagate into the later
stages and lead to an error build-up. In order to overcome this problem, we
propose a gradual source domain expansion (GSDE) algorithm. GSDE trains the UDA
task several times from scratch, each time reinitializing the network weights,
but each time expands the source dataset with target data. In particular, the
highest-scoring target data of the previous run are employed as pseudo-source
samples with their respective pseudo-label. Using this strategy, the
pseudo-source samples induce knowledge extracted from the previous run directly
from the start of the new training. This helps align the two domains better,
especially in the early training epochs. In this study, we first introduce a
strong baseline network and apply our GSDE strategy to it. We conduct
experiments and ablation studies on three benchmarks (Office-31, OfficeHome,
and DomainNet) and outperform state-of-the-art methods. We further show that
the proposed GSDE strategy can improve the accuracy of a variety of different
state-of-the-art UDA approaches.

通过渐进性的源领域扩展算法，将目标数据集作为伪源样本，从而通过多次重新初始化网络权重的训练，提取先前运行得出的知识，促进领域之间的对齐，改善无监督领域适应的准确性。

逐步扩展源领域的无监督领域自适应

Gradual Source Domain Expansion for Unsupervised Domain Adaptation

Transfer learning is a valuable tool in deep learning as it allows
propagating information from one "source dataset" to another "target dataset",
especially in the case of a small number of training examples in the latter.
Yet, discrepancies between the underlying distributions of the source and
target data are commonplace and are known to have a substantial impact on
algorithm performance. In this work we suggest novel information-theoretic
approaches for the analysis of the performance of deep neural networks in the
context of transfer learning. We focus on the task of semi-supervised transfer
learning, in which unlabeled samples from the target dataset are available
during network training on the source dataset. Our theory suggests that one may
improve the transferability of a deep neural network by incorporating
regularization terms on the target data based on information-theoretic
quantities, namely the Mutual Information and the Lautum Information. We
demonstrate the effectiveness of the proposed approaches in various
semi-supervised transfer learning experiments.

本文提出了一种新颖的信息论方法，以改善基于半监督学习的转移学习中深度神经网络的传递性，特别是在源数据集和目标数据集之间存在分歧的情况下，通过引入 Mutual Information 和 Lautum Information 来 应用于目标数据集的正则化项

一个基于信息理论的半监督迁移学习方法

An information-Theoretic Approach to Semi-supervised Transfer Learning

Cross-modal knowledge distillation deals with transferring knowledge from a
model trained with superior modalities (Teacher) to another model trained with
weak modalities (Student). Existing approaches require paired training examples
exist in both modalities. However, accessing the data from superior modalities
may not always be feasible. For example, in the case of 3D hand pose
estimation, depth maps, point clouds, or stereo images usually capture better
hand structures than RGB images, but most of them are expensive to be
collected. In this paper, we propose a novel scheme to train the Student in a
Target dataset where the Teacher is unavailable. Our key idea is to generalize
the distilled cross-modal knowledge learned from a Source dataset, which
contains paired examples from both modalities, to the Target dataset by
modeling knowledge as priors on parameters of the Student. We name our method
"Cross-Modal Knowledge Generalization" and demonstrate that our scheme results
in competitive performance for 3D hand pose estimation on standard benchmark
datasets.

本文提出了一种名为 “跨模态知识普适” 的新方法，将一个以高级模态（教师）训练的模型中的知识传递给以较弱模态（学生）训练的另一个模型，同时不需要在教师数据集中配对数据。通过在参数上建模知识，来将源数据集中积累的跨模态知识泛化到目标数据集，最终在标准基准数据集上展示了 3D 手势估计的竞争性表现。