Automatic radiology report generation has been an attracting research problem
towards computer-aided diagnosis to alleviate the workload of doctors in recent
years. Deep learning techniques for natural image captioning are successfully
adapted to generating radiology reports. However, radiology image reporting is
different from the natural image captioning task in two aspects: 1) the
accuracy of positive disease keyword mentions is critical in radiology image
reporting in comparison to the equivalent importance of every single word in a
natural image caption; 2) the evaluation of reporting quality should focus more
on matching the disease keywords and their associated attributes instead of
counting the occurrence of N-gram. Based on these concerns, we propose to
utilize a pre-constructed graph embedding module (modeled with a graph
convolutional neural network) on multiple disease findings to assist the
generation of reports in this work. The incorporation of knowledge graph allows
for dedicated feature learning for each disease finding and the relationship
modeling between them. In addition, we proposed a new evaluation metric for
radiology image reporting with the assistance of the same composed graph.
Experimental results demonstrate the superior performance of the methods
integrated with the proposed graph embedding module on a publicly accessible
dataset (IU-RR) of chest radiographs compared with previous approaches using
both the conventional evaluation metrics commonly adopted for image captioning
and our proposed ones.

本文介绍了使用预构建图嵌入模块（图卷积神经网络建模）来辅助放射图像报告生成的方法，该方法可以对多种疾病发现进行专门的特征学习并建立它们之间的关系，同时提出了用于医学图像报告的新的评估指标，并在公开数据集（IU-RR）上展示了该方法的优异性能。