Label scarcity is a bottleneck for improving task performance in specialised
domains. We propose a novel compositional transfer learning framework (DoT5 -
domain compositional zero-shot T5) for zero-shot domain transfer. Without
access to in-domain labels, DoT5 jointly learns domain knowledge (from MLM of
unlabelled in-domain free text) and task knowledge (from task training on more
readily available general-domain data) in a multi-task manner. To improve the
transferability of task training, we design a strategy named NLGU: we
simultaneously train NLG for in-domain label-to-data generation which enables
data augmentation for self-finetuning and NLU for label prediction. We evaluate
DoT5 on the biomedical domain and the resource-lean subdomain of radiology,
focusing on NLI, text summarisation and embedding learning. DoT5 demonstrates
the effectiveness of compositional transfer learning through multi-task
learning. In particular, DoT5 outperforms the current SOTA in zero-shot
transfer by over 7 absolute points in accuracy on RadNLI. We validate DoT5 with
ablations and a case study demonstrating its ability to solve challenging NLI
examples requiring in-domain expertise.

提出了一种新颖的组合式迁移学习框架（DoT5），将领域知识和任务知识进行多任务学习，解决了在特定领域提高任务性能的标签稀缺问题，并实现了零样本学习的目的。通过在生物医学领域和影像学的资源极少领域中进行了评估，特别是在 RadNLI 上，DoT5 在零样本转移方面的效果超过当前的 SOTA 7 个百分点以上。

基于文本到文本模型的零样本领域转移合成

Compositional Zero-Shot Domain Transfer with Text-to-Text Models

Open-vocabulary scene understanding aims to localize and recognize unseen
categories beyond the annotated label space. The recent breakthrough of 2D
open-vocabulary perception is largely driven by Internet-scale paired
image-text data with rich vocabulary concepts. However, this success cannot be
directly transferred to 3D scenarios due to the inaccessibility of large-scale
3D-text pairs. To this end, we propose to distill knowledge encoded in
pre-trained vision-language (VL) foundation models through captioning
multi-view images from 3D, which allows explicitly associating 3D and
semantic-rich captions. Further, to foster coarse-to-fine visual-semantic
representation learning from captions, we design hierarchical 3D-caption pairs,
leveraging geometric constraints between 3D scenes and multi-view images.
Finally, by employing contrastive learning, the model learns language-aware
embeddings that connect 3D and text for open-vocabulary tasks. Our method not
only remarkably outperforms baseline methods by 25.8% $\sim$ 44.7% hIoU and
14.5% $\sim$ 50.4% hAP$_{50}$ in open-vocabulary semantic and instance
segmentation, but also shows robust transferability on challenging zero-shot
domain transfer tasks. See the project website at
this https URL

通过对具有语义丰富标题的多视图图像进行记录，来设计分层三维标题对，使用对比学习，学习与图像相连的语言感知嵌入，并在开放词汇语义和实例分割方面表现出卓越的性能，具有鲁棒的可迁移性。