Radiology report generation (RRG) methods often lack sufficient medical knowledge to produce clinically accurate reports. The scene graph contains rich information to describe the objects in an image. We explore enriching the medical knowledge for RRG via a scene graph, which has not been done in the current RRG literature. To this end, we propose the Scene Graph aided RRG (SGRRG) network, a framework that generates region-level visual features, predicts anatomical attributes, and leverages an automatically generated scene graph, thus achieving medical knowledge distillation in an end-to-end manner. SGRRG is composed of a dedicated scene graph encoder responsible for translating the scene graph, and a scene graph-aided decoder that takes advantage of both patch-level and region-level visual information. A fine-grained, sentence-level attention method is designed to better dis-till the scene graph information. Extensive experiments demonstrate that SGRRG outperforms previous state-of-the-art methods in report generation and can better capture abnormal findings.

通过使用场景图来丰富医学知识，本研究提出了一种基于场景图的放射学报告生成网络（SGRRG网络），该网络能够生成区域级别的视觉特征，预测解剖属性，并利用自动生成的场景图，以实现端到端的医学知识提炼，并在报告生成方面胜过之前的最先进方法，并能更好地捕捉异常发现。

场景图辅助放射学报告生成