User simulators (USs) are commonly used to train task-oriented dialogue
systems (DSs) via reinforcement learning. The interactions often take place on
semantic level for efficiency, but there is still a gap from semantic actions
to natural language, which causes a mismatch between training and deployment
environment. Incorporating a natural language generation (NLG) module with USs
during training can partly deal with this problem. However, since the policy
and NLG of USs are optimised separately, these simulated user utterances may
not be natural enough in a given context. In this work, we propose a generative
transformer-based user simulator (GenTUS). GenTUS consists of an
encoder-decoder structure, which means it can optimise both the user policy and
natural language generation jointly. GenTUS generates both semantic actions and
natural language utterances, preserving interpretability and enhancing language
variation. In addition, by representing the inputs and outputs as word
sequences and by using a large pre-trained language model we can achieve
generalisability in feature representation. We evaluate GenTUS with automatic
metrics and human evaluation. Our results show that GenTUS generates more
natural language and is able to transfer to an unseen ontology in a zero-shot
fashion. In addition, its behaviour can be further shaped with reinforcement
learning opening the door to training specialised user simulators.

本文提出了一个基于生成式变形器的用户模拟器（GenTUS），它可以同时优化用户策略和自然语言生成，并生成语义动作和自然语言话语，保持可解释性和增强语言变异。它还利用大型预训练语言模型，将输入和输出表示为单词序列，从而实现特征表示的可泛化性。评估结果表明，GenTUS 生成的自然语言更真实，能够在没有先验知识的情况下转移到未见过的本体，并且可以通过强化学习进一步优化行为，从而为训练专门的用户模拟器打开了大门。