Radiology report generation aims to automatically generate detailed and
coherent descriptive reports alongside radiology images. Previous work mainly
focused on refining fine-grained image features or leveraging external
knowledge. However, the precise alignment of fine-grained image features with
corresponding text descriptions has not been considered. This paper presents a
novel method called Fine-grained Image-Text Aligner (FITA) to construct
fine-grained alignment for image and text features. It has three novel designs:
Image Feature Refiner (IFR), Text Feature Refiner (TFR) and Contrastive Aligner
(CA). IFR and TFR aim to learn fine-grained image and text features,
respectively. We achieve this by leveraging saliency maps to effectively fuse
symptoms with corresponding abnormal visual regions, and by utilizing a
meticulously constructed triplet set for training. Finally, CA module aligns
fine-grained image and text features using contrastive loss for precise
alignment. Results show that our method surpasses existing methods on the
widely used benchmark

本文介绍了一种名为 Fine-grained Image-Text Aligner (FITA) 的新方法，用于构建图像和文本特征的精细对齐，包括图像特征细化器（IFR）、文本特征细化器（TFR）和对比对齐器（CA），通过利用显著性图来有效融合症状与相应的异常视觉区域，以及利用精心构造的三元组集进行训练，最终使用对比损失实现精确定位的图像和文本特征的对齐。结果表明，我们的方法在广泛使用的基准测试上超过了现有的方法。

放射学报告生成的细粒度图像文本对齐器

FITA: Fine-grained Image-Text Aligner for Radiology Report Generation

Data scarcity and privacy concerns limit the availability of high-quality
medical images for public use, which can be mitigated through medical image
synthesis. However, current medical image synthesis methods often struggle to
accurately capture the complexity of detailed anatomical structures and
pathological conditions. To address these challenges, we propose a novel
medical image synthesis model that leverages fine-grained image-text alignment
and anatomy-pathology prompts to generate highly detailed and accurate
synthetic medical images. Our method integrates advanced natural language
processing techniques with image generative modeling, enabling precise
alignment between descriptive text prompts and the synthesized images'
anatomical and pathological details. The proposed approach consists of two key
components: an anatomy-pathology prompting module and a fine-grained
alignment-based synthesis module. The anatomy-pathology prompting module
automatically generates descriptive prompts for high-quality medical images. To
further synthesize high-quality medical images from the generated prompts, the
fine-grained alignment-based synthesis module pre-defines a visual codebook for
the radiology dataset and performs fine-grained alignment between the codebook
and generated prompts to obtain key patches as visual clues, facilitating
accurate image synthesis. We validate the superiority of our method through
experiments on public chest X-ray datasets and demonstrate that our synthetic
images preserve accurate semantic information, making them valuable for various
medical applications.

通过细粒度的图像 - 文本对齐和解剖病理提示，我们提出了一种新颖的医学图像合成模型，能够生成高度详细和准确的合成医学图像，我们验证了我们的方法在公共胸部 X 光数据集上的优越性，并展示了我们的合成图像保留准确的语义信息，对各种医学应用具有价值。