It is important to define meaningful and interpretable automatic evaluation
metrics for open-domain dialog research. Standard language generation metrics
have been shown to be ineffective for dialog. This paper introduces the FED
metric (fine-grained evaluation of dialog), an automatic evaluation metric
which uses DialoGPT, without any fine-tuning or supervision. It also introduces
the FED dataset which is constructed by annotating a set of human-system and
human-human conversations with eighteen fine-grained dialog qualities. The FED
metric (1) does not rely on a ground-truth response, (2) does not require
training data and (3) measures fine-grained dialog qualities at both the turn
and whole dialog levels. FED attains moderate to strong correlation with human
judgement at both levels.

本文介绍了 FED 度量（对话的细粒度评估），该度量使用 DialoGPT，不需要微调或监督，以及 FED 数据集，称为人机和人人对话的十八个细粒度对话质量的注释构成。FED 度量不依赖于真实回答，不需要训练数据，并且在回合和整个对话水平上测量细粒度对话质量。FED 在两个级别上与人类判断具有中度到强度的相关性。