This paper presents Domain-Specific Sub-network (DoSS). It uses a set of
masks obtained through pruning to define a sub-network for each domain and
finetunes the sub-network parameters on domain data. This performs very closely
and drastically reduces the number of parameters compared to finetuning the
whole network on each domain. Also a method to make masks unique per domain is
proposed and shown to greatly improve the generalization to unseen domains. In
our experiments on German to English machine translation the proposed method
outperforms the strong baseline of continue training on multi-domain (medical,
tech and religion) data by 1.47 BLEU points. Also continue training DoSS on new
domain (legal) outperforms the multi-domain (medical, tech, religion, legal)
baseline by 1.52 BLEU points.

本文提出了领域专属子网（DoSS）方法，利用剪枝技术得到一组掩码，为每个领域定义一个子网络并在领域数据上微调子网络参数，使其性能接近且参数数量大大减少。同时提出了一种使掩码在每个领域中唯一的方法，大大提高了到未知领域的泛化能力。在德语至英语机器翻译实验中，DoSS 方法的表现优于多个领域（医学、技术和宗教）数据的继续训练的强基线方法 1.47 BLEU 点。而在新领域（法律）上继续训练 DoSS，其表现则比多个领域（医学、技术、宗教、法律）数据的基线方法高出 1.52 BLEU 点。