In many applications, the labeled data at the learner's disposal is subject
to privacy constraints and is relatively limited. To derive a more accurate
predictor for the target domain, it is often beneficial to leverage publicly
available labeled data from an alternative domain, somewhat close to the target
domain. This is the modern problem of supervised domain adaptation from a
public source to a private target domain. We present two $(\epsilon,
\delta)$-differentially private adaptation algorithms for supervised
adaptation, for which we make use of a general optimization problem, recently
shown to benefit from favorable theoretical learning guarantees. Our first
algorithm is designed for regression with linear predictors and shown to solve
a convex optimization problem. Our second algorithm is a more general solution
for loss functions that may be non-convex but Lipschitz and smooth. While our
main objective is a theoretical analysis, we also report the results of several
experiments first demonstrating that the non-private versions of our algorithms
outperform adaptation baselines and next showing that, for larger values of the
target sample size or $\epsilon$, the performance of our private algorithms
remains close to that of the non-private formulation.

通过使用一种新的优化问题来训练，本文提出了两种隐私保护的监督适应算法，分别适用于具有线性预测器的回归和非凸但利普希兹平滑的损失函数，并且在实验中表明这两种算法的效果表现优于基线算法。

具有理论保证的差分隐私域自适应

Differentially Private Domain Adaptation with Theoretical Guarantees

Self-Supervised Learning (SSL) has allowed leveraging large amounts of
unlabeled speech data to improve the performance of speech recognition models
even with small annotated datasets. Despite this, speech SSL representations
may fail while facing an acoustic mismatch between the pretraining and target
datasets. To address this issue, we propose a novel supervised domain
adaptation method, designed for cases exhibiting such a mismatch in acoustic
domains. It consists in applying properly calibrated data augmentations on a
large clean dataset, bringing it closer to the target domain, and using it as
part of an initial fine-tuning stage. Augmentations are automatically selected
through the minimization of a conditional-dependence estimator, based on the
target dataset. The approach is validated during an oracle experiment with
controlled distortions and on two amateur-collected low-resource domains,
reaching better performances compared to the baselines in both cases.

本篇论文提出一种用于有声学领域差异的监督领域自适应方法，通过数据增强来减小源域和目标域的差异，并在初步微调阶段使用，证明了该方法的有效性。

自监督语音表示域适应微调的自动数据增强

Automatic Data Augmentation for Domain Adapted Fine-Tuning of  Self-Supervised Speech Representations

This work provides a framework for addressing the problem of supervised
domain adaptation with deep models. The main idea is to exploit adversarial
learning to learn an embedded subspace that simultaneously maximizes the
confusion between two domains while semantically aligning their embedding. The
supervised setting becomes attractive especially when there are only a few
target data samples that need to be labeled. In this few-shot learning
scenario, alignment and separation of semantic probability distributions is
difficult because of the lack of data. We found that by carefully designing a
training scheme whereby the typical binary adversarial discriminator is
augmented to distinguish between four different classes, it is possible to
effectively address the supervised adaptation problem. In addition, the
approach has a high speed of adaptation, i.e. it requires an extremely low
number of labeled target training samples, even one per category can be
effective. We then extensively compare this approach to the state of the art in
domain adaptation in two experiments: one using datasets for handwritten digit
recognition, and one using datasets for visual object recognition.

采用对抗训练来学习嵌入的子空间，同时最大化两个域之间的混淆并在语义上对齐它们，解决了少量有标签目标数据样本的深度模型的有监督域自适应问题。

少样本对抗性域自适应

Few-Shot Adversarial Domain Adaptation

We investigate techniques for supervised domain adaptation for neural machine
translation where an existing model trained on a large out-of-domain dataset is
adapted to a small in-domain dataset. In this scenario, overfitting is a major
challenge. We investigate a number of techniques to reduce overfitting and
improve transfer learning, including regularization techniques such as dropout
and L2-regularization towards an out-of-domain prior. In addition, we introduce
tuneout, a novel regularization technique inspired by dropout. We apply these
techniques, alone and in combination, to neural machine translation, obtaining
improvements on IWSLT datasets for English->German and English->Russian. We
also investigate the amounts of in-domain training data needed for domain
adaptation in NMT, and find a logarithmic relationship between the amount of
training data and gain in BLEU score.

本研究探讨了神经机器翻译的监督域自适应技术，针对现有模型在大型非领域数据集训练后，如何适应小型领域数据集所出现的过拟合问题。在整个过程中，正则化技术，如 dropout 和 L2 正则化与非领域先验之间的关系得到了深入研究。此外，文章提出了一种新型的正则化技术 ——tuneout，即启发式 dropout。我们将这些技术单独或结合应用于神经机器翻译，以英语 -> 德语、英语 -> 俄语的 IWSLT 数据集为例取得了改进。此外，我们还研究了 NMT 领域中所需的领域内训练数据量，并发现培训数据与 BLEU 分数之间存在对数关系。