Evaluating radiology reports is a challenging problem as factual correctness
is extremely important due to the need for accurate medical communication about
medical images. Existing automatic evaluation metrics either suffer from
failing to consider factual correctness (e.g., BLEU and ROUGE) or are limited
in their interpretability (e.g., F1CheXpert and F1RadGraph). In this paper, we
introduce GREEN (Generative Radiology Report Evaluation and Error Notation), a
radiology report generation metric that leverages the natural language
understanding of language models to identify and explain clinically significant
errors in candidate reports, both quantitatively and qualitatively. Compared to
current metrics, GREEN offers: 1) a score aligned with expert preferences, 2)
human interpretable explanations of clinically significant errors, enabling
feedback loops with end-users, and 3) a lightweight open-source method that
reaches the performance of commercial counterparts. We validate our GREEN
metric by comparing it to GPT-4, as well as to error counts of 6 experts and
preferences of 2 experts. Our method demonstrates not only higher correlation
with expert error counts, but simultaneously higher alignment with expert
preferences when compared to previous approaches."

介绍了一种名为 GREEN（生成式放射学报告评估和错误标注）的放射学报告生成度量方法，利用语言模型的自然语言理解能力定量和定性地识别和解释候选报告中的临床显著错误，该方法不仅与专家错误计数具有更高的相关性，而且与专家偏好更加一致。