Multiple different responses are often plausible for a given open domain
dialog context. Prior work has shown the importance of having multiple valid
reference responses for meaningful and robust automated evaluations. In such
cases, common practice has been to collect more human written references.
However, such collection can be expensive, time consuming, and not easily
scalable. Instead, we propose a novel technique for automatically expanding a
human generated reference to a set of candidate references. We fetch plausible
references from knowledge sources, and adapt them so that they are more fluent
in context of the dialog instance in question. More specifically, we use (1) a
commonsense knowledge base to elicit a large number of plausible reactions
given the dialog history (2) relevant instances retrieved from dialog corpus,
using similar past as well as future contexts. We demonstrate that our
automatically expanded reference sets lead to large improvements in
correlations of automated metrics with human ratings of system outputs for
DailyDialog dataset.

本文提出了一种新技术，即自动将人工生成的参考文本扩展为一组备选参考文本的技术，该技术通过从知识源中获取可信的文本响应并对其进行调整，以使其在当前对话情境中更加符合逻辑，说明其方法可以显著提高与人工系统输出评级的自动指标的相关性。