We propose CHRT (Control Hidden Representation Transformation) - a controlled
language generation framework that steers large language models to generate
text pertaining to certain attributes (such as toxicity). CHRT gains attribute
control by modifying the hidden representation of the base model through
learned transformations. We employ a contrastive-learning framework to learn
these transformations that can be combined to gain multi-attribute control. The
effectiveness of CHRT is experimentally shown by comparing it with seven
baselines over three attributes. CHRT outperforms all the baselines in the task
of detoxification, positive sentiment steering, and text simplification while
minimizing the loss in linguistic qualities. Further, our approach has the
lowest inference latency of only 0.01 seconds more than the base model, making
it the most suitable for high-performance production environments. We
open-source our code and release two novel datasets to further propel
controlled language generation research.

提出了一种名为 CHRT 的控制隐藏表征转换框架，可以通过修改基础模型的隐藏表征来控制翻译的属性，通过对比七个基线模型，实验结果表明 CHRT 在解决文本毒性、正面情感导向和文本简化方面的效果优于其他方法，并且保持了语言质量，在推断延迟方面也表现最佳，可适用于高性能生产环境中。