Medical imaging is frequently used in clinical practice and trials for
diagnosis and treatment. Writing imaging reports is time-consuming and can be
error-prone for inexperienced radiologists. Therefore, automatically generating
radiology reports is highly desired to lighten the workload of radiologists and
accordingly promote clinical automation, which is an essential task to apply
artificial intelligence to the medical domain. In this paper, we propose to
generate radiology reports with memory-driven Transformer, where a relational
memory is designed to record key information of the generation process and a
memory-driven conditional layer normalization is applied to incorporating the
memory into the decoder of Transformer. Experimental results on two prevailing
radiology report datasets, IU X-Ray and MIMIC-CXR, show that our proposed
approach outperforms previous models with respect to both language generation
metrics and clinical evaluations. Particularly, this is the first work
reporting the generation results on MIMIC-CXR to the best of our knowledge.
Further analyses also demonstrate that our approach is able to generate long
reports with necessary medical terms as well as meaningful image-text attention
mappings.

本文提出了使用记忆驱动 Transformer 来生成放射学报告，实验证明该方法可以以更高的质量、更长的长度和更多的医学术语产生放射学报告，此为我们所知第一次在 MIMIC-CXR 上实现放射学报告的生成结果。