Automatic radiology report generation is challenging as medical images or
reports are usually similar to each other due to the common content of anatomy.
This makes a model hard to capture the uniqueness of individual images and is
prone to producing undesired generic or mismatched reports. This situation
calls for learning more discriminative features that could capture even
fine-grained mismatches between images and reports. To achieve this, this paper
proposes a novel framework to learn discriminative image and report features by
distinguishing them from their closest peers, i.e., hard negatives. Especially,
to attain more discriminative features, we gradually raise the difficulty of
such a learning task by creating increasingly hard negative reports for each
image in the feature space during training, respectively. By treating the
increasingly hard negatives as auxiliary variables, we formulate this process
as a min-max alternating optimisation problem. At each iteration, conditioned
on a given set of hard negative reports, image and report features are learned
as usual by minimising the loss functions related to report generation. After
that, a new set of harder negative reports will be created by maximising a loss
reflecting image-report alignment. By solving this optimisation, we attain a
model that can generate more specific and accurate reports. It is noteworthy
that our framework enhances discriminative feature learning without introducing
extra network weights. Also, in contrast to the existing way of generating hard
negatives, our framework extends beyond the granularity of the dataset by
generating harder samples out of the training set. Experimental study on
benchmark datasets verifies the efficacy of our framework and shows that it can
serve as a plug-in to readily improve existing medical report generation
models.

本文提出了一种新的框架，通过逐步提高每个图像在特征空间中的更难负面反馈的难度，从而学习区分性图像和报告特征，以生成更特定和准确的医疗报告，实验证明这种方法可以作为插件轻松地改进现有的医学报告生成模型。