Conventional Domain Adaptation (DA) methods aim to learn domain-invariant
feature representations to improve the target adaptation performance. However,
we motivate that domain-specificity is equally important since in-domain
trained models hold crucial domain-specific properties that are beneficial for
adaptation. Hence, we propose to build a framework that supports
disentanglement and learning of domain-specific factors and task-specific
factors in a unified model. Motivated by the success of vision transformers in
several multi-modal vision problems, we find that queries could be leveraged to
extract the domain-specific factors. Hence, we propose a novel
Domain-specificity-inducing Transformer (DSiT) framework for disentangling and
learning both domain-specific and task-specific factors. To achieve
disentanglement, we propose to construct novel Domain-Representative Inputs
(DRI) with domain-specific information to train a domain classifier with a
novel domain token. We are the first to utilize vision transformers for domain
adaptation in a privacy-oriented source-free setting, and our approach achieves
state-of-the-art performance on single-source, multi-source, and multi-target
benchmarks

在隐私导向的无源情况下，我们提出了一种使用视觉转换器进行领域适应的方法，通过构建领域可辨识输入（DRI），利用查询提取特定领域信息，并在统一模型中实现领域特异性和任务特异性的解耦和学习，实现了在单源、多源和多目标基准上的最先进性能。

领域特异性引导变形器以实现无来源领域适应

Domain-Specificity Inducing Transformers for Source-Free Domain  Adaptation

Although large neural language models (LMs) like BERT can be finetuned to
yield state-of-the-art results on many NLP tasks, it is often unclear what
these models actually learn. Here we study using such LMs to fill in entities
in human-authored comparative questions, like ``Which country is older, India
or ______?'' -- i.e., we study the ability of neural LMs to ask (not answer)
reasonable questions. We show that accuracy in this fill-in-the-blank task is
well-correlated with human judgements of whether a question is reasonable, and
that these models can be trained to achieve nearly human-level performance in
completing comparative questions in three different subdomains. However,
analysis shows that what they learn fails to model any sort of broad notion of
which entities are semantically comparable or similar -- instead the trained
models are very domain-specific, and performance is highly correlated with
co-occurrences between specific entities observed in the training set. This is
true both for models that are pretrained on general text corpora, as well as
models trained on a large corpus of comparison questions. Our study thus
reinforces recent results on the difficulty of making claims about a deep
model's world knowledge or linguistic competence based on performance on
specific benchmark problems. We make our evaluation datasets publicly available
to foster future research on complex understanding and reasoning in such models
at standards of human interaction.

本论文研究以大型神经语言模型填充人类编写的比较问题中的实体，证明在三个不同子域中这些模型的性能可以接近于人类的完成水平，但分析表明它们学习的内容并不能建模任何广泛的可比较或类似的实体，而是非常特定于特定领域，性能与训练集中特定实体的共现高度相关。