Recent language models have been improved by the addition of external memory.
Nearest neighbor language models retrieve similar contexts to assist in word
prediction. The addition of locality levels allows a model to learn how to
weight neighbors based on their relative location to the current text in source
documents, and have been shown to further improve model performance. Nearest
neighbor models have been explored for controllable generation but have not
examined the use of locality levels. We present a novel approach for this
purpose and evaluate it using automatic and human evaluation on politeness,
formality, supportiveness, and toxicity textual data. We find that our model is
successfully able to control style and provides a better fluency-style
trade-off than previous work.

使用外部存储器改进的最近邻语言模型，通过检索相似的语境来辅助词语预测，添加局部级别使模型能够学习如何根据相对于源文件中当前文本的位置来加权邻居，从而进一步提高模型性能。我们提出了一种新的方法，并在礼貌、正式、支持性和毒性文本数据上进行自动和人工评估，发现我们的模型能够成功控制风格，并提供比以往更好的流利度 - 风格权衡。