The ultimate goal of dialog research is to develop systems that can be
effectively used in interactive settings by real users. To this end, we
introduced the Interactive Evaluation of Dialog Track at the 9th Dialog System
Technology Challenge. This track consisted of two sub-tasks. The first sub-task
involved building knowledge-grounded response generation models. The second
sub-task aimed to extend dialog models beyond static datasets by assessing them
in an interactive setting with real users. Our track challenges participants to
develop strong response generation models and explore strategies that extend
them to back-and-forth interactions with real users. The progression from
static corpora to interactive evaluation introduces unique challenges and
facilitates a more thorough assessment of open-domain dialog systems. This
paper provides an overview of the track, including the methodology and results.
Furthermore, it provides insights into how to best evaluate open-domain dialog
models

该论文介绍了针对开放领域对话系统的交互式评估方法，为开发基于知识的响应生成模型，探索将其扩展到与真实用户的交互中提供挑战，从而介绍了从静态语料库到交互式评估的进展，以及如何最好评估开放领域对话模型的见解。

DSTC9 中的对话跟踪交互式评估

Interactive Evaluation of Dialog Track at DSTC9

Current Knowledge-Grounded Dialogue Generation (KDG) models specialize in
producing rational and factual responses. However, to establish long-term
relationships with users, the KDG model needs the capability to generate
responses in a desired style or attribute. Thus, we study a new problem:
Stylized Knowledge-Grounded Dialogue Generation (SKDG). It presents two
challenges: (1) How to train a SKDG model where no <context, knowledge,
stylized response> triples are available. (2) How to cohere with context and
preserve the knowledge when generating a stylized response. In this paper, we
propose a novel disentangled template rewriting (DTR) method which generates
responses via combing disentangled style templates (from monolingual stylized
corpus) and content templates (from KDG corpus). The entire framework is
end-to-end differentiable and learned without supervision. Extensive
experiments on two benchmarks indicate that DTR achieves a significant
improvement on all evaluation metrics compared with previous state-of-the-art
stylized dialogue generation methods. Besides, DTR achieves comparable
performance with the state-of-the-art KDG methods in standard KDG evaluation
setting.

本研究提出了一种基于 knowledge-grounded 的对话生成模型，称为 Disentangled Template Rewriting（DTR）模型，它能够生成符合预期风格的对话，且整个框架是可学习的，并在两个基准测试中得到了显著改进的结果。