The automation of writing imaging reports is a valuable tool for alleviating
the workload of radiologists. Crucial steps in this process involve the
cross-modal alignment between medical images and reports, as well as the
retrieval of similar historical cases. However, the presence of
presentation-style vocabulary (e.g., sentence structure and grammar) in reports
poses challenges for cross-modal alignment. Additionally, existing methods for
similar historical cases retrieval face suboptimal performance owing to the
modal gap issue. In response, this paper introduces a novel method, named
Factual Serialization Enhancement (FSE), for chest X-ray report generation. FSE
begins with the structural entities approach to eliminate presentation-style
vocabulary in reports, providing specific input for our model. Then, uni-modal
features are learned through cross-modal alignment between images and factual
serialization in reports. Subsequently, we present a novel approach to retrieve
similar historical cases from the training set, leveraging aligned image
features. These features implicitly preserve semantic similarity with their
corresponding reference reports, enabling us to calculate similarity solely
among aligned features. This effectively eliminates the modal gap issue for
knowledge retrieval without the requirement for disease labels. Finally, the
cross-modal fusion network is employed to query valuable information from these
cases, enriching image features and aiding the text decoder in generating
high-quality reports. Experiments on MIMIC-CXR and IU X-ray datasets from both
specific and general scenarios demonstrate the superiority of FSE over
state-of-the-art approaches in both natural language generation and clinical
efficacy metrics.

该论文介绍了一种名为 Factual Serialization Enhancement (FSE) 的新型方法，通过结构实体方法来消除报告中的演示风格词汇，实现胸部 X 光报告生成，并通过交叉模态对齐学习单模态特征以及从训练集中检索类似历史案例，最终利用交叉模态融合网络从这些案例中查询有价值的信息，提高图像特征和文本解码器生成高质量报告的能力。该方法在 MIMIC-CXR 和 IU X-ray 数据集上的实验证明了其在自然语言生成和临床效果度量方面优于现有方法。