In recent years, the growing demand for medical imaging diagnosis has brought
a significant burden to radiologists. The existing Med-VLP methods provide a
solution for automated medical image analysis which learns universal
representations from large-scale medical images and reports and benefits
downstream tasks without requiring fine-grained annotations. However, the
existing methods based on joint image-text reconstruction neglect the
importance of cross-modal alignment in conjunction with joint reconstruction,
resulting in inadequate cross-modal interaction. In this paper, we propose a
unified Med-VLP framework based on Multi-task Paired Masking with Alignment
(MPMA) to integrate the cross-modal alignment task into the joint image-text
reconstruction framework to achieve more comprehensive cross-modal interaction,
while a global and local alignment (GLA) module is designed to assist
self-supervised paradigm in obtaining semantic representations with rich domain
knowledge. To achieve more comprehensive cross-modal fusion, we also propose a
Memory-Augmented Cross-Modal Fusion (MA-CMF) module to fully integrate visual
features to assist in the process of report reconstruction. Experimental
results show that our approach outperforms previous methods over all downstream
tasks, including uni-modal, cross-modal and multi-modal tasks.

该研究提出了一种新的医学图像分析解决方案：基于多任务配对掩模对齐（MPMA）的统一 Med-VLP 框架，其中设计了全局和局部对齐（GLA）模块和记忆增强型跨模态融合（MA-CMF）模块，以实现更全面的跨模态交互，并在全部下游任务中优于以前的方法。