Recent approaches to controlled text generation (CTG) often involve
manipulating the weights or logits of base language models (LMs) at decoding
time. However, these methods are inapplicable to latest black-box LMs and
ineffective at preserving the core semantics of the base LM's original
generations. In this work, we propose Locate&Edit(L&E), an efficient and
flexible energy-based approach to CTG, which edits text outputs from a base LM
using off-the-shelf energy models. Given text outputs from the base LM, L&E
first locates spans that are most relevant to constraints (e.g., toxicity)
utilizing energy models, and then edits these spans by replacing them with more
suitable alternatives. Importantly, our method is compatible with black-box
LMs, as it requires only the text outputs. Also, since L&E doesn't mandate
specific architecture for its component models, it can work with a diverse
combination of available off-the-shelf models. Moreover, L&E preserves the base
LM's original generations, by selectively modifying constraint-related aspects
of the texts and leaving others unchanged. These targeted edits also ensure
that L&E operates efficiently. Our experiments confirm that L&E achieves
superior semantic preservation of the base LM generations and speed, while
simultaneously obtaining competitive or improved constraint satisfaction.
Furthermore, we analyze how the granularity of energy distribution impacts CTG
performance and find that fine-grained, regression-based energy models improve
constraint satisfaction, compared to conventional binary classifier energy
models.

通过利用黑盒语言模型的文本输出，以生成的文本中最相关的部分为约束进行编辑，Locate&Edit (L&E) 提出了一种有效而灵活的能量模型方法，既能保留基本语言模型的原始生成结果，又能满足相关的约束条件。

Locate&Edit: 基于能量的文本编辑技术用于高效、灵活和真实的可控文本生成

Locate&Edit: Energy-based Text Editing for Efficient, Flexible, and  Faithful Controlled Text Generation

Retrieval-augmented language models have exhibited promising performance
across various areas of natural language processing (NLP), including
fact-critical tasks. However, due to the black-box nature of advanced large
language models (LLMs) and the non-retrieval-oriented supervision signal of
specific tasks, the training of retrieval model faces significant challenges
under the setting of black-box LLM. We propose an approach leveraging
Fine-grained Feedback with Reinforcement Retrieval (FFRR) to enhance
fact-checking on news claims by using black-box LLM. FFRR adopts a two-level
strategy to gather fine-grained feedback from the LLM, which serves as a reward
for optimizing the retrieval policy, by rating the retrieved documents based on
the non-retrieval ground truth of the task. We evaluate our model on two public
datasets for real-world news claim verification, and the results demonstrate
that FFRR achieves significant improvements over strong LLM-enabled and non-LLM
baselines.

使用黑盒语言模型的精细反馈和增强检索，增强了新闻索赔事实验证的结果，并在真实世界的数据集上显著改进了强大的语言模型和非语言模型基线。