Medical vision-language pre-training has emerged as a promising approach for learning domain-general representations of medical image and text. Current algorithms that exploit the global and local alignment between medical image and text could however be marred by the redundant information in medical data. To address this issue, we propose a grounded knowledge-enhanced medical vision-language pre-training (GK-MVLP) framework for chest X-ray. In this framework, medical knowledge is grounded to the appropriate anatomical regions by using a transformer-based grounded knowledge-enhanced module for fine-grained alignment between anatomical region-level visual features and the textural features of medical knowledge. The performance of GK-MVLP is competitive with or exceeds the state of the art on downstream chest X-ray disease classification, disease localization, report generation, and medical visual question-answering tasks. Our results show the advantage of incorporating grounding mechanism to remove biases and improve the alignment between chest X-ray image and radiology report.

我们提出了一种基于Transformer的基于地面知识增强模块的医学视觉-语言预训练（GK-MVLP）框架，通过细粒度对应医学知识的解剖区域级视觉特征和文本特征，将医学知识地面化到适当的解剖区域，优化胸部X光图像和放射学报告的对齐，并在下游任务上展现出与或超过现有技术的竞争力，包括胸部X光疾病分类、疾病定位、报告生成和医学视觉问答。结果显示了引入地面化机制消除偏见并改善胸部X光图像和放射学报告的对齐的优势。

基于知识增强的医学CXR可视化语言预训练模型