Medical report generation demands automatic creation of coherent and precise
descriptions for medical images. However, the scarcity of labelled medical
image-report pairs poses formidable challenges in developing large-scale neural
networks capable of harnessing the potential of artificial intelligence,
exemplified by large language models. This study builds upon the
state-of-the-art vision-language pre-training and fine-tuning approach, BLIP-2,
to customize general large-scale foundation models. Integrating adapter tuning
and a medical knowledge enhancement loss, our model significantly improves
accuracy and coherence. Validation on the dataset of ImageCLEFmedical 2023
demonstrates our model's prowess, achieving the best-averaged results against
several state-of-the-art methods. Significant improvements in ROUGE and CIDEr
underscore our method's efficacy, highlighting promising outcomes for the rapid
medical-domain adaptation of the vision-language foundation models in
addressing challenges posed by data scarcity.

医疗报告生成需要自动创建连贯准确的医学图像描述。本研究基于最先进的视觉语言预训练和微调方法 BLIP-2，利用适配器调优和医学知识增强损失，显著提高了模型的准确性和连贯性。在 ImageCLEFmedical 2023 的数据集上验证，我们的模型取得了最佳的平均结果，优于几种最先进的方法。ROUGE 和 CIDEr 的显著改进突显了我们方法的有效性，为快速适应视觉语言基础模型以应对数据匮乏的挑战带来了有希望的结果。