Controllable text generation (CTG) aims to generate text with desired
attributes, and decoding-time-based methods have shown promising performance on
this task. However, in this paper, we identify the phenomenon of Attribute
Collapse for the first time. It causes the fluency of generated text to rapidly
decrease when the control strength exceeds a critical value, rendering the text
completely unusable. This limitation hinders the effectiveness of decoding
methods in achieving high levels of controllability. To address this problem,
we propose a novel lightweight decoding framework named Air-Decoding. Its main
idea is reconstructing the attribute distributions to balance the weights
between attribute words and non-attribute words to generate more fluent text.
Specifically, we train prefixes by prefix-tuning to obtain attribute
distributions. Then we design a novel attribute distribution reconstruction
method to balance the obtained distributions and use the reconstructed
distributions to guide language models for generation, effectively avoiding the
issue of Attribute Collapse. Experiments on multiple CTG tasks prove that our
method achieves a new state-of-the-art control performance.

我们提出了一种名为 Air-Decoding 的新型轻量级解码框架，该框架通过重构属性分布来平衡属性单词和非属性单词之间的权重，从而生成更流畅的文本，有效避免了属性崩塌问题，实验结果表明我们的方法实现了最先进的控制性能。