Diagnostic Captioning (DC) automatically generates a diagnostic text from one
or more medical images (e.g., X-rays, MRIs) of a patient. Treated as a draft,
the generated text may assist clinicians, by providing an initial estimation of
the patient's condition, speeding up and helping safeguard the diagnostic
process. The accuracy of a diagnostic text, however, strongly depends on how
well the key medical conditions depicted in the images are expressed. We
propose a new data-driven guided decoding method that incorporates medical
information, in the form of existing tags capturing key conditions of the
image(s), into the beam search of the diagnostic text generation process. We
evaluate the proposed method on two medical datasets using four DC systems that
range from generic image-to-text systems with CNN encoders and RNN decoders to
pre-trained Large Language Models. The latter can also be used in few- and
zero-shot learning scenarios. In most cases, the proposed mechanism improves
performance with respect to all evaluation measures. We provide an open-source
implementation of the proposed method at this https URL

诊断字幕自动生成一份诊断性文本，从一个或多个患者的医学图像（如 X 射线、MRI）中。本论文提出了一种新的数据驱动引导解码方法，将医学信息纳入诊断文本生成过程的波束搜索中，并在两个医学数据集上使用四个诊断字幕系统进行评估。在大多数情况下，所提出的机制在所有评估指标上提高了性能。

基于数据驱动的诊断字幕引导解码机制

A Data-Driven Guided Decoding Mechanism for Diagnostic Captioning

Purpose: This study focuses on the development of automated text generation
from radiology images, termed diagnostic captioning, to assist medical
professionals in reducing clinical errors and improving productivity. The aim
is to provide tools that enhance report quality and efficiency, which can
significantly impact both clinical practice and deep learning research in the
biomedical field. Methods: In our participation in the ImageCLEFmedical2024
Caption evaluation campaign, we explored caption prediction tasks using
advanced Transformer-based models. We developed methods incorporating
Transformer encoder-decoder and Query Transformer architectures. These models
were trained and evaluated to generate diagnostic captions from radiology
images. Results: Experimental evaluations demonstrated the effectiveness of our
models, with the VisionDiagnostor-BioBART model achieving the highest BERTScore
of 0.6267. This performance contributed to our team, DarkCow, achieving third
place on the leaderboard. Conclusion: Our diagnostic captioning models show
great promise in aiding medical professionals by generating high-quality
reports efficiently. This approach can facilitate better data processing and
performance optimization in medical imaging departments, ultimately benefiting
healthcare delivery.

本篇研究旨在利用自动化文本生成技术从放射学图像中提取诊断说明，以辅助医疗专业人员减少临床错误并提高工作效率，目的是提供既能够增强报告质量和效率的工具，也能够对临床实践和生物医学领域的深度学习研究产生重大影响的方法。