Diagnostic Captioning (DC) automatically generates a diagnostic text from one
or more medical images (e.g., X-rays, MRIs) of a patient. Treated as a draft,
the generated text may assist clinicians, by providing an initial estimation of
the patient's condition, speeding up and helping safeguard the diagnostic
process. The accuracy of a diagnostic text, however, strongly depends on how
well the key medical conditions depicted in the images are expressed. We
propose a new data-driven guided decoding method that incorporates medical
information, in the form of existing tags capturing key conditions of the
image(s), into the beam search of the diagnostic text generation process. We
evaluate the proposed method on two medical datasets using four DC systems that
range from generic image-to-text systems with CNN encoders and RNN decoders to
pre-trained Large Language Models. The latter can also be used in few- and
zero-shot learning scenarios. In most cases, the proposed mechanism improves
performance with respect to all evaluation measures. We provide an open-source
implementation of the proposed method at this https URL

诊断字幕自动生成一份诊断性文本，从一个或多个患者的医学图像（如 X 射线、MRI）中。本论文提出了一种新的数据驱动引导解码方法，将医学信息纳入诊断文本生成过程的波束搜索中，并在两个医学数据集上使用四个诊断字幕系统进行评估。在大多数情况下，所提出的机制在所有评估指标上提高了性能。