Dialogue datasets are crucial for deep learning-based task-oriented dialogue
system research. While numerous English language multi-domain task-oriented
dialogue datasets have been developed and contributed to significant
advancements in task-oriented dialogue systems, such a dataset does not exist
in Japanese, and research in this area is limited compared to that in English.
In this study, towards the advancement of research and development of
task-oriented dialogue systems in Japanese, we constructed JMultiWOZ, the first
Japanese language large-scale multi-domain task-oriented dialogue dataset.
Using JMultiWOZ, we evaluated the dialogue state tracking and response
generation capabilities of the state-of-the-art methods on the existing major
English benchmark dataset MultiWOZ2.2 and the latest large language model
(LLM)-based methods. Our evaluation results demonstrated that JMultiWOZ
provides a benchmark that is on par with MultiWOZ2.2. In addition, through
evaluation experiments of interactive dialogues with the models and human
participants, we identified limitations in the task completion capabilities of
LLMs in Japanese.

为了推动日语任务导向对话系统的研究和发展，本研究构建了 JMultiWOZ，这是第一个规模较大的多领域任务导向对话数据集。通过使用 JMultiWOZ 评估英语基准数据集 MultiWOZ2.2 和最新的大型语言模型 (LLM) 方法的对话状态跟踪和响应生成能力，评估结果表明 JMultiWOZ 与 MultiWOZ2.2 具有相当的水平。此外，通过与模型和人类参与者进行交互对话的评估实验，我们发现日语中 LLMs 在任务完成能力上存在局限性。