The impression is crucial for the referring physicians to grasp key
information since it is concluded from the findings and reasoning of
radiologists. To alleviate the workload of radiologists and reduce repetitive
human labor in impression writing, many researchers have focused on automatic
impression generation. However, recent works on this task mainly summarize the
corresponding findings and pay less attention to the radiology images. In
clinical, radiographs can provide more detailed valuable observations to
enhance radiologists' impression writing, especially for complicated cases.
Besides, each sentence in findings usually focuses on single anatomy, so they
only need to be matched to corresponding anatomical regions instead of the
whole image, which is beneficial for textual and visual features alignment.
Therefore, we propose a novel anatomy-enhanced multimodal model to promote
impression generation. In detail, we first construct a set of rules to extract
anatomies and put these prompts into each sentence to highlight anatomy
characteristics. Then, two separate encoders are applied to extract features
from the radiograph and findings. Afterward, we utilize a contrastive learning
module to align these two representations at the overall level and use a
co-attention to fuse them at the sentence level with the help of
anatomy-enhanced sentence representation. Finally, the decoder takes the fused
information as the input to generate impressions. The experimental results on
two benchmark datasets confirm the effectiveness of the proposed method, which
achieves state-of-the-art results.

本文提出了一种新的解决方案来增强放射学家的结论写作，重点在于，将放射学图像和检查结果组合在一起生成结论，主要包括解剖学增强多模态模型和对比学习模块，经过两个基准数据集的实验证明，该方法显著优于其他自动结论生成模型。