Prompt learning with immensely large Casual Language Models (CLMs) has been
shown promising for attribute-controllable text generation (CTG). However,
vanilla prompt tuning tends to imitate training corpus characteristics beyond
the control attributes, resulting in a poor generalization ability. Moreover,
it is less able to capture the relationship between different attributes,
further limiting the control performance. In this paper, we propose a new CTG
approach, namely DisCup, which incorporates the attribute knowledge of
discriminator to optimize the control-prompts, steering a frozen CLM to produce
attribute-specific texts. Specifically, the frozen CLM model, capable of
producing multitudinous texts, is first used to generate the next-token
candidates based on the context, so as to ensure the diversity of tokens to be
predicted. Then, we leverage an attribute-discriminator to select
desired/undesired tokens from those candidates, providing the inter-attribute
knowledge. Finally, we bridge the above two traits by an unlikelihood objective
for prompt-tuning. Extensive experimental results show that DisCup can achieve
a new state-of-the-art control performance while maintaining an efficient and
high-quality text generation, only relying on around 10 virtual tokens.

本文提出了一种新的文本生成方法 —DisCup，该方法利用判别器的属性知识来优化控制提示，引导一个固定的 CLM 生成属性特定的文本，可以实现最新的控制性能并保持高效和高质量的文本生成。

DisCup: 判别器协作非可能性提示调整技术用于可控文本生成

DisCup: Discriminator Cooperative Unlikelihood Prompt-tuning for Controllable Text Generation

Negation is a core construction in natural language. Despite being very
successful on many tasks, state-of-the-art pre-trained language models often
handle negation incorrectly. To improve language models in this regard, we
propose to augment the language modeling objective with an unlikelihood
objective that is based on negated generic sentences from a raw text corpus. By
training BERT with the resulting combined objective we reduce the mean top~1
error rate to 4% on the negated LAMA dataset. We also see some improvements on
the negated NLI benchmarks.

本文针对自然语言中否定句的建构，借助负面通用语句的反向学习目标，优化了 BERT 语言模型，将 negated LAMA 数据集的平均 top1 错误率降至 4％，并在 negated NLI 基准测试中看到了一些改进。