Controllable text generation systems often leverage control codes to direct
various properties of the output like style and length. Inspired by recent work
on causal inference for NLP, this paper reveals a previously overlooked flaw in
these control code-based conditional text generation algorithms. Spurious
correlations in the training data can lead models to incorrectly rely on parts
of the input other than the control code for attribute selection, significantly
undermining downstream generation quality and controllability. We demonstrate
the severity of this issue with a series of case studies and then propose two
simple techniques to reduce these correlations in training sets. The first
technique is based on resampling the data according to an example's propensity
towards each linguistic attribute (IPS). The second produces multiple
counterfactual versions of each example and then uses an additional feedback
mechanism to remove noisy examples (feedback aware self-training, FAST). We
evaluate on 3 tasks -- news headline, meta review, and search ads generation --
and demonstrate that FAST can significantly improve the controllability and
language quality of generated outputs when compared to state-of-the-art
controllable text generation approaches.

这篇论文针对控制文本生成算法中可能存在的潜在问题，利用因果推断技术发现了训练数据中的关联错误，提出通过倾向得分采样和反事实版本生成机制消除这些关联，实验结果表明该方法在生成性能和语言品质上都得到了显著提升。