Controllable text generation is a growing field within natural language
generation (NLG) that focuses on producing text that meets specific constraints
in real-world applications. Previous approaches, such as plug-and-play
controllers (PPCs), aimed to steer the properties of generated text in a
flexible manner. However, these methods often compromised the integrity of the
language model's decoding process, resulting in less smooth text generation.
Alternatively, other techniques utilized multiple attribute prompts to align
the generated text with desired attributes, but this approach required prompt
design for each attribute and was dependent on the size of the language model.
This paper introduces a novel method for flexible attribute control in text
generation using pre-trained language models (PLMs). The proposed approach aims
to enhance the fluency of generated text by guiding the generation process with
PPCs. The key idea is to dynamically adjust the distribution of generated text
by modifying prompts, effectively constraining the output space of the language
model and influencing the desired attribute. To enable smooth cooperation
between the PLM and the PPC, our work innovatively proposes a new model
fine-tuning method: Reinforcement Learning with Dynamic Adjust Feedback
(RLDAF).This fine-tuning process adapts a small subset of the language model's
parameters based on the generating actions taken during the PPC control
process. The resulting harmonious collaboration between the PLM and PPC leads
to improved smoothness in text generation during inference. Extensive
experiments were conducted on the SST2 dataset, and the proposed method
outperformed previous approaches in various evaluation metrics, including text
fluency and attribute consistency.

提出了一种新颖的使用预训练语言模型进行灵活属性控制的文本生成方法，通过动态调整生成文本的分布，修改提示信息，限制语言模型的输出空间并影响所需属性，在 PLM 和 PPC 之间实现协作，通过 RLDAF 方法在推理过程中改进文本生成的顺畅性，实验证明该方法在多个评估指标下（包括文本流畅性和属性一致性）优于之前的方法。