The remarkable generative capabilities of multimodal foundation models are
currently being explored for a variety of applications. Generating radiological
impressions is a challenging task that could significantly reduce the workload
of radiologists. In our study we explored and analyzed the generative abilities
of GPT-4 for Chest X-ray impression generation. To generate and evaluate
impressions of chest X-rays based on different input modalities (image, text,
text and image), a blinded radiological report was written for 25-cases of the
publicly available NIH-dataset. GPT-4 was given image, finding section or both
sequentially to generate an input dependent impression. In a blind randomized
reading, 4-radiologists rated the impressions and were asked to classify the
impression origin (Human, AI), providing justification for their decision.
Lastly text model evaluation metrics and their correlation with the
radiological score (summation of the 4 dimensions) was assessed. According to
the radiological score, the human-written impression was rated highest,
although not significantly different to text-based impressions. The automated
evaluation metrics showed moderate to substantial correlations to the
radiological score for the image impressions, however individual scores were
highly divergent among inputs, indicating insufficient representation of
radiological quality. Detection of AI-generated impressions varied by input and
was 61% for text-based impressions. Impressions classified as AI-generated had
significantly worse radiological scores even when written by a radiologist,
indicating potential bias. Our study revealed significant discrepancies between
a radiological assessment and common automatic evaluation metrics depending on
the model input. The detection of AI-generated findings is subject to bias that
highly rated impressions are perceived as human-written.

通过评估 GPT-4 对于胸部 X 射线的生成能力，本研究探讨了基于多模态基础模型的放射学印象生成的潜力，发现放射学评估与常见自动评估指标存在显著差异，并暴露了 AI 生成结果存在潜在偏见的问题。

评估 GPT-4 对胸部 X 射线诊断印象生成的效果：基于性能和认知的读者研究

Evaluation of GPT-4 for chest X-ray impression generation: A reader  study on performance and perception

Radiology reports play a critical role in communicating medical findings to
physicians. In each report, the impression section summarizes essential
radiology findings. In clinical practice, writing impression is highly demanded
yet time-consuming and prone to errors for radiologists. Therefore, automatic
impression generation has emerged as an attractive research direction to
facilitate such clinical practice. Existing studies mainly focused on
introducing salient word information to the general text summarization
framework to guide the selection of the key content in radiology findings.
However, for this task, a model needs not only capture the important words in
findings but also accurately describe their relations so as to generate
high-quality impressions. In this paper, we propose a novel method for
automatic impression generation, where a word graph is constructed from the
findings to record the critical words and their relations, then a Word Graph
guided Summarization model (WGSum) is designed to generate impressions with the
help of the word graph. Experimental results on two datasets, OpenI and
MIMIC-CXR, confirm the validity and effectiveness of our proposed approach,
where the state-of-the-art results are achieved on both datasets. Further
experiments are also conducted to analyze the impact of different graph designs
to the performance of our method.

本文提出了一种新颖的自动印象生成方法，其中从发现中构建词图以记录关键词及其关系，然后设计了一个词图引导的总结模型 (WGSum) 来生成印象。实验结果表明，我们的方法在两个数据集上取得了最先进的结果，这进一步证实了我们的方法的有效性和可行性。