Source-free domain adaptation aims to adapt a source model trained on
fully-labeled source domain data to a target domain with unlabeled target
domain data. Source data is assumed inaccessible due to proprietary or privacy
reasons. Existing works use the source model to pseudolabel target data, but
the pseudolabels are unreliable due to data distribution shift between source
and target domain. In this work, we propose to leverage an ImageNet pre-trained
feature extractor in a new co-learning framework to improve target pseudolabel
quality for finetuning the source model. Benefits of the ImageNet feature
extractor include that it is not source-biased and it provides an alternate
view of features and classification decisions different from the source model.
Such pre-trained feature extractors are also publicly available, which allows
us to readily leverage modern network architectures that have strong
representation learning ability. After co-learning, we sharpen predictions of
non-pseudolabeled samples by entropy minimization. Evaluation on 3 benchmark
datasets show that our proposed method can outperform existing source-free
domain adaptation methods, as well as unsupervised domain adaptation methods
which assume joint access to source and target data.

本文研究基于 ImageNet 预训练特征提取器的共同学习框架，以提高面向目标域的伪标记质量并进行迁移学习，同时使用熵最小化来提高非伪标记样本的预测精度，并在三个基准数据集上进行评估。结果显示，该方法优于现有的面向源自由域适应方法和假设联合访问源和目标数据的无监督域适应方法。

使用预训练网络进行协同学习，改善无源域适应

Co-Learning with Pre-Trained Networks Improves Source-Free Domain  Adaptation

Semi-supervised learning is the problem of training an accurate predictive
model by combining a small labeled dataset with a presumably much larger
unlabeled dataset. Many methods for semi-supervised deep learning have been
developed, including pseudolabeling, consistency regularization, and
contrastive learning techniques. Pseudolabeling methods however are highly
susceptible to confounding, in which erroneous pseudolabels are assumed to be
true labels in early iterations, thereby causing the model to reinforce its
prior biases and thereby fail to generalize to strong predictive performance.
We present a new approach to suppress confounding errors through a method we
describe as Semi-supervised Contrastive Outlier removal for Pseudo Expectation
Maximization (SCOPE). Like basic pseudolabeling, SCOPE is related to
Expectation Maximization (EM), a latent variable framework which can be
extended toward understanding cluster-assumption deep semi-supervised
algorithms. However, unlike basic pseudolabeling which fails to adequately take
into account the probability of the unlabeled samples given the model, SCOPE
introduces an outlier suppression term designed to improve the behavior of EM
iteration given a discrimination DNN backbone in the presence of outliers. Our
results show that SCOPE greatly improves semi-supervised classification
accuracy over a baseline, and furthermore when combined with consistency
regularization achieves the highest reported accuracy for the semi-supervised
CIFAR-10 classification task using 250 and 4000 labeled samples. Moreover, we
show that SCOPE reduces the prevalence of confounding errors during
pseudolabeling iterations by pruning erroneous high-confidence pseudolabeled
samples that would otherwise contaminate the labeled set in subsequent
retraining iterations.

该研究提出了一种名为 SCOPE 的半监督对比异常值去除伪期望最大化方法，旨在压制伪标签带来的混淆错误，改进基于 EM 迭代的半监督深度学习算法行为，在移除离群值后，SCOPE 同时还能够有效提高半监督学习的分类精度，进一步搭配一致性正则化实现了基于半监督 CIFAR-10 分类任务在 250 和 4000 个标记样本下的最高分类精度。

基于半监督对比度的离群值去除算法：伪最大期望算法 (SCOPE)

Semi-supervised Contrastive Outlier removal for Pseudo Expectation Maximization (SCOPE)

Unlike traditional supervised learning, in many settings only partial
feedback is available. We may only observe outcomes for the chosen actions, but
not the counterfactual outcomes associated with other alternatives. Such
settings encompass a wide variety of applications including pricing, online
marketing and precision medicine. A key challenge is that observational data
are influenced by historical policies deployed in the system, yielding a biased
data distribution. We approach this task as a domain adaptation problem and
propose a self-training algorithm which imputes outcomes with categorical
values for finite unseen actions in the observational data to simulate a
randomized trial through pseudolabeling, which we refer to as Counterfactual
Self-Training (CST). CST iteratively imputes pseudolabels and retrains the
model. In addition, we show input consistency loss can further improve CST
performance which is shown in recent theoretical analysis of pseudolabeling. We
demonstrate the effectiveness of the proposed algorithms on both synthetic and
real datasets.

本文提出了一种基于自训练和伪标记的方法，将观察数据中的有限未见行动的结果用分类值进行插值，以模拟随机试验的结果，并解决传统监督学习不可用的问题，应用到计价、在线营销、精准医疗等领域。