We propose a Distributional Approach for addressing Controlled Text
Generation from pre-trained Language Models (LMs). This approach permits to
specify, in a single formal framework, both "pointwise" and "distributional"
constraints over the target LM -- to our knowledge, the first model with such
generality -- while minimizing KL divergence from the initial LM distribution.
The optimal target distribution is then uniquely determined as an explicit EBM
(Energy-Based Model) representation. From that optimal representation we then
train a target controlled Autoregressive LM through an adaptive distributional
variant of Policy Gradient. We conduct a first set of experiments over
pointwise constraints showing the advantages of our approach over a set of
baselines, in terms of obtaining a controlled LM balancing constraint
satisfaction with divergence from the initial LM. We then perform experiments
over distributional constraints, a unique feature of our approach,
demonstrating its potential as a remedy to the problem of Bias in Language
Models. Through an ablation study, we show the effectiveness of our adaptive
technique for obtaining faster convergence. (Code available at
this https URL)

本文提出一种基于分布式方法的控制文本生成方案，旨在在单个形式框架中指定 “点对点” 和 “分布式” 约束，实现与初始 LM 分布 KL 散度最小的目标 LM 的训练。同时，基于 Energy-Based Model 表示确定最优目标分布。该方案通过自适应分布式模型改进了 Policy Gradient. 作者在实验中表明，该方案在满足约束方面较其它方法具有优势。实验结果还表明该方案应对了语言模型中 “偏见” 的问题。