Knowledge Grounded Conversation Models (KGCM) are usually based on a
selection/retrieval module and a generation module, trained separately or
simultaneously, with or without having access to a gold knowledge option. With
the introduction of large pre-trained generative models, the selection and
generation part have become more and more entangled, shifting the focus towards
enhancing knowledge incorporation (from multiple sources) instead of trying to
pick the best knowledge option. These approaches however depend on knowledge
labels and/or a separate dense retriever for their best performance. In this
work we study the unsupervised selection abilities of pre-trained generative
models (e.g. BART) and show that by adding a score-and-aggregate module between
encoder and decoder, they are capable of learning to pick the proper knowledge
through minimising the language modelling loss (i.e. without having access to
knowledge labels). Trained as such, our model - K-Mine - shows competitive
selection and generation performance against models that benefit from knowledge
labels and/or separate dense retriever.

本文研究了预训练生成模型的无监督选择能力，通过在编码器和解码器之间添加得分和聚合模块。这个模型 - K-Mine - 在没有知识标签和 / 或分离的密集检索器的情况下展示了有竞争力的选择和生成性能。