Current language models achieve low perplexity but their resulting
generations still suffer from toxic responses, repetitiveness and
contradictions. The standard language modeling setup fails to address these
issues. In this paper, we introduce a new architecture, {\sc Director}, that
consists of a unified generator-classifier with both a language modeling and a
classification head for each output token. Training is conducted jointly using
both standard language modeling data, and data labeled with desirable and
undesirable sequences. Experiments in several settings show that the model has
competitive training and decoding speed compared to standard language models
while yielding superior results, alleviating known issues while maintaining
generation quality. It also outperforms existing model guiding approaches in
terms of both accuracy and efficiency.

本文介绍了一个新的基于统一生成器 - 分类器框架的 Director 语言模型，该模型结合语言建模和分类学习，并使用包括有利和不利序列标记的数据进行训练，实验证明该模型相较于标准语言模型可以大幅减少毒瘤响应、重复性、矛盾等问题，在保持生成质量的同时，训练和解码速度也相对较快。此外，该模型在准确性和效率方面均优于现有的模型指导方法。