Existing reference-free metrics have obvious limitations for evaluating controlled text generation models. Unsupervised metrics can only provide a task-agnostic evaluation result which correlates weakly with human judgments, whereas supervised ones may overfit task-specific data with poor generalization ability to other datasets. In this paper, we propose an unsupervised reference-free metric called CTRLEval, which evaluates controlled text generation from different aspects by formulating each aspect into multiple text infilling tasks. On top of these tasks, the metric assembles the generation probabilities from a pre-trained language model without any model training. Experimental results show that our metric has higher correlations with human judgments than other baselines, while obtaining better generalization of evaluating generated texts from different models and with different qualities.

本文提出了一种称为 CTRLEval 的无监督无参考文本生成度量方法，通过将每个方面制定为多个文本填充任务，从不同角度评估受控制的文本生成。此度量方法无需进行任何模型训练，只需从预训练的语言模型中收集生成概率。实验结果表明，相比其他基线，该度量方法与人类判断的相关性更高，同时能更好地评估不同质量和模型的生成文本的一般性。

CTRLEval：用于评估受控文本生成的无监督无参考度量