We present a unified Vision-Language pretrained Model (VLMo) that jointly
learns a dual encoder and a fusion encoder with a modular Transformer network.
Specifically, we introduce Mixture-of-Modality-Experts (MoME) Transformer,
where each block contains a pool of modality-specific experts and a shared
self-attention layer. Because of the modeling flexibility of MoME, pretrained
VLMo can be fine-tuned as a fusion encoder for vision-language classification
tasks, or used as a dual encoder for efficient image-text retrieval. Moreover,
we propose a stagewise pre-training strategy, which effectively leverages
large-scale image-only and text-only data besides image-text pairs.
Experimental results show that VLMo achieves state-of-the-art results on
various vision-language tasks, including VQA, NLVR2 and image-text retrieval.
The code and pretrained models are available at this https URL

本研究提出了统一的视觉 - 语言预训练模型 (VLMo)，通过模块化的 Transformer 网络共同学习双编码器和融合编码器。实验结果表明，VLMo 在各种视觉 - 语言任务中取得了最先进的结果。