Beyond the common difficulties faced in the natural image captioning, medical
report generation specifically requires the model to describe a medical image
with a fine-grained and semantic-coherence paragraph that should satisfy both
medical commonsense and logic. Previous works generally extract the global
image features and attempt to generate a paragraph that is similar to
referenced reports; however, this approach has two limitations. Firstly, the
regions of primary interest to radiologists are usually located in a small area
of the global image, meaning that the remainder parts of the image could be
considered as irrelevant noise in the training procedure. Secondly, there are
many similar sentences used in each medical report to describe the normal
regions of the image, which causes serious data bias. This deviation is likely
to teach models to generate these inessential sentences on a regular basis. To
address these problems, we propose an Auxiliary Signal-Guided Knowledge
Encoder-Decoder (ASGK) to mimic radiologists' working patterns. In more detail,
ASGK integrates internal visual feature fusion and external medical linguistic
information to guide medical knowledge transfer and learning. The core
structure of ASGK consists of a medical graph encoder and a natural language
decoder, inspired by advanced Generative Pre-Training (GPT). Experiments on the
CX-CHR dataset and our COVID-19 CT Report dataset demonstrate that our proposed
ASGK is able to generate a robust and accurate report, and moreover outperforms
state-of-the-art methods on both medical terminology classification and
paragraph generation metrics.

该论文提出了一种基于图像特征和医疗语言信息的 Auxiliary Signal-Guided Knowledge Encoder-Decoder (ASGK) 模型，用于解决医学报告生成中的一些问题，并在 CX-CHR 数据集和 COVID-19 CT 报告数据集上取得了良好效果。