An effective multi-turn instruction-following assistant can be developed by
creating a simulator that can generate useful interaction data. Apart from
relying on its intrinsic weights, an ideal user simulator should also be able
to bootstrap external knowledge rapidly in its raw form to simulate the
multifarious diversity of text available over the internet. Previous user
simulators generally lacked diversity, were mostly closed domain, and
necessitated rigid schema making them inefficient to rapidly scale to
incorporate external knowledge. In this regard, we introduce, Kaucus, a
Knowledge-Augmented User Simulator framework, to outline a process of creating
diverse user simulators, that can seamlessly exploit external knowledge as well
as benefit downstream assistant model training. Through two GPT-J based
simulators viz., a Retrieval Augmented Simulator and a Summary Controlled
Simulator we generate diverse simulator-assistant interactions. Through reward
and preference model-based evaluations, we find that these interactions serve
as useful training data and create more helpful downstream assistants. We also
find that incorporating knowledge through retrieval augmentation or summary
control helps create better assistants.

通过引入 Kaucus，一种知识增强的用户模拟器框架，我们使用基于 GPT-J 的检索增强模拟器和摘要控制模拟器生成多样化的模拟器－助手交互数据，进而发现这些交互数据作为有用的训练数据，可以创建更好的下游助手。同时，我们发现通过检索增强或摘要控制的方式融入知识有助于创建更好的助手。