Medical image visual question answering (VQA) is a task to answer clinical
questions, given a radiographic image, which is a challenging problem that
requires a model to integrate both vision and language information. To solve
medical VQA problems with a limited number of training data, pretrain-finetune
paradigm is widely used to improve the model generalization. In this paper, we
propose a self-supervised method that applies Masked image modeling, Masked
language modeling, Image text matching and Image text alignment via contrastive
learning (M2I2) for pretraining on medical image caption dataset, and finetunes
to downstream medical VQA tasks. The proposed method achieves state-of-the-art
performance on all the three public medical VQA datasets. Our codes and models
are available at this https URL.

本文介绍了一种自我监督方法 - 对遮蔽图像建模、遮蔽语言建模、图像文本匹配和图像文本对齐进行对比学习的 M2I2 方法，应用于医学图像字幕数据集的预训练，并对下游医学 VQA 任务进行微调。该方法在三个公共医学 VQA 数据集上实现了最先进的性能。