Existing mainstream approaches follow the encoder-decoder paradigm for
generating radiology reports. They focus on improving the network structure of
encoders and decoders, which leads to two shortcomings: overlooking the
modality gap and ignoring report content constraints. In this paper, we
proposed Textual Inversion and Self-supervised Refinement (TISR) to address the
above two issues. Specifically, textual inversion can project text and image
into the same space by representing images as pseudo words to eliminate the
cross-modeling gap. Subsequently, self-supervised refinement refines these
pseudo words through contrastive loss computation between images and texts,
enhancing the fidelity of generated reports to images. Notably, TISR is
orthogonal to most existing methods, plug-and-play. We conduct experiments on
two widely-used public datasets and achieve significant improvements on various
baselines, which demonstrates the effectiveness and generalization of TISR. The
code will be available soon.

此篇论文提出了基于文本反转和自监督改进 (TISR) 的方法，通过解决编码器和解码器网络结构的问题，克服了模态差异和报告内容限制，实现了文本和图像在同一空间中的表示，通过对图像和文本之间的对比损失计算来改进生成的报告的准确性。该方法与现有方法独立且易于使用，通过在两个常用公开数据集上的实验，取得了显著的改进效果，展示了 TISR 的有效性和普适性。