Radiology report generation aims to automatically generate detailed and
coherent descriptive reports alongside radiology images. Previous work mainly
focused on refining fine-grained image features or leveraging external
knowledge. However, the precise alignment of fine-grained image features with
corresponding text descriptions has not been considered. This paper presents a
novel method called Fine-grained Image-Text Aligner (FITA) to construct
fine-grained alignment for image and text features. It has three novel designs:
Image Feature Refiner (IFR), Text Feature Refiner (TFR) and Contrastive Aligner
(CA). IFR and TFR aim to learn fine-grained image and text features,
respectively. We achieve this by leveraging saliency maps to effectively fuse
symptoms with corresponding abnormal visual regions, and by utilizing a
meticulously constructed triplet set for training. Finally, CA module aligns
fine-grained image and text features using contrastive loss for precise
alignment. Results show that our method surpasses existing methods on the
widely used benchmark

本文介绍了一种名为 Fine-grained Image-Text Aligner (FITA) 的新方法，用于构建图像和文本特征的精细对齐，包括图像特征细化器（IFR）、文本特征细化器（TFR）和对比对齐器（CA），通过利用显著性图来有效融合症状与相应的异常视觉区域，以及利用精心构造的三元组集进行训练，最终使用对比损失实现精确定位的图像和文本特征的对齐。结果表明，我们的方法在广泛使用的基准测试上超过了现有的方法。