Large language models (LLMs), such as ChatGPT, have demonstrated impressive
capabilities in various tasks and attracted an increasing interest as a natural
language interface across many domains. Recently, large vision-language models
(VLMs) like BLIP-2 and GPT-4 have been intensively investigated, which learn
rich vision-language correlation from image-text pairs. However, despite these
developments, the application of LLMs and VLMs in image quality assessment
(IQA), particularly in medical imaging, remains to be explored, which is
valuable for objective performance evaluation and potential supplement or even
replacement of radiologists' opinions. To this end, this paper introduces
IQAGPT, an innovative image quality assessment system integrating an image
quality captioning VLM with ChatGPT for generating quality scores and textual
reports. First, we build a CT-IQA dataset for training and evaluation,
comprising 1,000 CT slices with diverse quality levels professionally
annotated. To better leverage the capabilities of LLMs, we convert annotated
quality scores into semantically rich text descriptions using a prompt
template. Second, we fine-tune the image quality captioning VLM on the CT-IQA
dataset to generate quality descriptions. The captioning model fuses the image
and text features through cross-modal attention. Third, based on the quality
descriptions, users can talk with ChatGPT to rate image quality scores or
produce a radiological quality report. Our preliminary results demonstrate the
feasibility of assessing image quality with large models. Remarkably, our
IQAGPT outperforms GPT-4 and CLIP-IQA, as well as the multi-task classification
and regression models that solely rely on images.

在医学成像中，本研究以 IQAGPT 为例，结合了图像质量说明的 VLM 和 ChatGPT，利用大型语言模型实现图像质量评估和生成文本报告。结果表明 IQAGPT 在图像质量评估方面表现优异，超过了 GPT-4、CLIP-IQA 和仅依赖图像的多任务分类和回归模型。