In recent years, deep neural networks (DNNs) trained with transformed data
have been applied to various applications such as privacy-preserving learning,
access control, and adversarial defenses. However, the use of transformed data
decreases the performance of models. Accordingly, in this paper, we propose a
novel method for fine-tuning models with transformed images under the use of
the vision transformer (ViT). The proposed domain adaptation method does not
cause the accuracy degradation of models, and it is carried out on the basis of
the embedding structure of ViT. In experiments, we confirmed that the proposed
method prevents accuracy degradation even when using encrypted images with the
CIFAR-10 and CIFAR-100 datasets.

我们提出了一种使用视觉变换器 (ViT) 对转换图像进行模型微调的新方法，该转换图像域自适应方法不会导致模型准确度下降，基于 ViT 的嵌入结构进行实施，实验证实了该方法在使用 CIFAR-10 和 CIFAR-100 数据集的加密图像时仍能防止准确度下降。

加密图像下高效微调视觉转换模型的领域适应

Domain Adaptation for Efficiently Fine-tuning Vision Transformer with  Encrypted Images

Intermediate training of pre-trained transformer-based language models on
domain-specific data leads to substantial gains for downstream tasks. To
increase efficiency and prevent catastrophic forgetting alleviated from full
domain-adaptive pre-training, approaches such as adapters have been developed.
However, these require additional parameters for each layer, and are criticized
for their limited expressiveness. In this work, we introduce TADA, a novel
task-agnostic domain adaptation method which is modular, parameter-efficient,
and thus, data-efficient. Within TADA, we retrain the embeddings to learn
domain-aware input representations and tokenizers for the transformer encoder,
while freezing all other parameters of the model. Then, task-specific
fine-tuning is performed. We further conduct experiments with meta-embeddings
and newly introduced meta-tokenizers, resulting in one model per task in
multi-domain use cases. Our broad evaluation in 4 downstream tasks for 14
domains across single- and multi-domain setups and high- and low-resource
scenarios reveals that TADA is an effective and efficient alternative to full
domain-adaptive pre-training and adapters for domain adaptation, while not
introducing additional parameters or complex training steps.

本文提出了一种名为 TADA 的新颖方法，用于领域自适应，其具有模块化、参数高效和数据高效的优点，并且与完全域自适应预训练和适配器相比，具有相同或更好的性能而无需引入额外的参数或复杂的训练步骤。

TADA: 面向 Transformer 的高效任务无关领域自适应

TADA: Efficient Task-Agnostic Domain Adaptation for Transformers

Knowledge of the changing traffic is critical in risk management. Customs
offices worldwide have traditionally relied on local resources to accumulate
knowledge and detect tax fraud. This naturally poses countries with weak
infrastructure to become tax havens of potentially illicit trades. The current
paper proposes DAS, a memory bank platform to facilitate knowledge sharing
across multi-national customs administrations to support each other. We propose
a domain adaptation method to share transferable knowledge of frauds as
prototypes while safeguarding the local trade information. Data encompassing
over 8 million import declarations have been used to test the feasibility of
this new system, which shows that participating countries may benefit up to
2-11 times in fraud detection with the help of shared knowledge. We discuss
implications for substantial tax revenue potential and strengthened policy
against illicit trades.

本论文提出了一种名为 DAS 的记忆库平台，旨在促进多国海关部门之间的知识共享，以支持彼此。我们提出了一种领域自适应方法，共享作为原型的欺诈行为的可转移知识，同时保护本地贸易信息。通过测试超过 800 万个进口申报数据，发现参与国家可以通过共享知识获得 2-11 倍的欺诈检测收益。该系统具有实现可观的税收潜力和加强打击非法交易政策的意义。