Despite the success of Large Language Models (LLMs) on various tasks
following human instructions, controlling model generation at inference time
poses a persistent challenge. In this paper, we introduce Ctrl-G, an adaptable
framework that facilitates tractable and flexible control of LLM generation to
reliably follow logical constraints. Ctrl-G combines any production-ready LLM
with a Hidden Markov Model, enabling LLM outputs to adhere to logical
constraints represented as deterministic finite automata. We show that Ctrl-G,
when applied to a TULU2-7B model, outperforms GPT3.5 and GPT4 on the task of
interactive text editing: specifically, for the task of generating text
insertions/continuations following logical constraints, Ctrl-G achieves over
30% higher satisfaction rate in human evaluation compared to GPT4. When applied
to medium-size language models (e.g., GPT2-large), Ctrl-G also beats its
counterparts for constrained generation by large margins on standard
benchmarks. Additionally, as a proof-of-concept study, we experiment Ctrl-G on
the Grade School Math benchmark to assist LLM reasoning, foreshadowing the
application of Ctrl-G, as well as other constrained generation approaches,
beyond traditional language generation tasks.

通过引入适应性框架 Ctrl-G，该研究提出了一种可实现对大型语言模型的可控生成的方法，并将其应用于交互文本编辑和有限状态自动机表示的逻辑约束，结果显示 Ctrl-G 在人类评估中相比于 GPT4 以及 GPT3.5 取得了 30% 以上的满意率提升，并在标准基准测试中超越了中型语言模型。此外，该研究还进行了 Grade School Math 基准测试作为概念验证，展示了 Ctrl-G 在辅助大型语言模型推理方面的潜力。

大型语言模型的适应性逻辑控制

Adaptable Logical Control for Large Language Models

The incredible generative ability of large-scale text-to-image (T2I) models
has demonstrated strong power of learning complex structures and meaningful
semantics. However, relying solely on text prompts cannot fully take advantage
of the knowledge learned by the model, especially when flexible and accurate
controlling (e.g., color and structure) is needed. In this paper, we aim to
``dig out" the capabilities that T2I models have implicitly learned, and then
explicitly use them to control the generation more granularly. Specifically, we
propose to learn simple and lightweight T2I-Adapters to align internal
knowledge in T2I models with external control signals, while freezing the
original large T2I models. In this way, we can train various adapters according
to different conditions, achieving rich control and editing effects in the
color and structure of the generation results. Further, the proposed
T2I-Adapters have attractive properties of practical value, such as
composability and generalization ability. Extensive experiments demonstrate
that our T2I-Adapter has promising generation quality and a wide range of
applications.

该论文提出了使用轻量级的 T2I 适配器来对图像生成过程进行更加精准的控制，实现在不同条件下的色彩和结构控制，并具有可组合性和广泛的应用前景。

T2I-Adapter: 学习适配器以挖掘更多可控能力的文本到图像扩散模型

T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models

Transformer-based pre-trained language models boost the performance of
open-domain dialogue systems. Prior works leverage Transformer-based
pre-trained language models to generate texts with desired attributes in two
general approaches: (1) gradient-based methods: updating all latent
representations of pre-trained models with gradients from attribute models; (2)
weighted-decoding methods: re-ranking beam candidates from pre-trained models
with attribute functions. However, gradient-based methods lead to high
computation cost and can easily get overfitted on small training sets, while
weighted-decoding methods are inherently constrained by the low-variance
high-bias pre-trained model. In this work, we propose a novel approach to
control the generation of Transformer-based pre-trained language models: the
SideControl framework, which leverages a novel control attributes loss to
incorporate useful control signals, and is shown to perform well with very
limited training samples. We evaluate our proposed method on two benchmark
open-domain dialogue datasets, and results show that the SideControl framework
has better controllability, higher generation quality and better
sample-efficiency than existing gradient-based and weighted-decoding baselines.

该论文提出了一种名为 SideControl 框架的新方法，该方法可以控制 Transformer 预训练语言模型的生成，具有更好的可控性、更高的生成质量和更好的样本效率，适用于极少量的训练数据。

SideControl: 通过 Additive Side Networks 控制开放域对话生成

SideControl: Controlled Open-domain Dialogue Generation via Additive  Side Networks

While large-scale language models (LMs) are able to imitate the distribution
of natural language well enough to generate realistic text, it is difficult to
control which regions of the distribution they generate. This is especially
problematic because datasets used for training large LMs usually contain
significant toxicity, hate, bias, and negativity. We propose GeDi as an
efficient method for using smaller LMs as generative discriminators to guide
generation from large LMs to make them safer and more controllable. GeDi guides
generation at each step by computing classification probabilities for all
possible next tokens via Bayes rule by normalizing over two class-conditional
distributions; one conditioned on the desired attribute, or control code, and
another conditioned on the undesired attribute, or anti control code. We find
that GeDi gives stronger controllability than the state of the art method while
also achieving generation speeds more than 30 times faster. Additionally,
training GeDi on only four topics allows us to controllably generate new topics
zero-shot from just a keyword, unlocking a new capability that previous
controllable generation methods do not have. Lastly, we show that GeDi can make
GPT-2 (1.5B parameters) significantly less toxic without sacrificing linguistic
quality, making it by far the most practical existing method for detoxifying
large language models while maintaining a fast generation speed.

本论文提出了一种名为 GeDi 的方法，通过引入更小的 LM 作为生成鉴别器以指导在大型 LM 上更安全和可控的生成。GeDi 计算每个标记的分类概率，以便于实现对生成的控制，并且可以使 GPT-2 在不影响语言质量的情况下显著降低毒性，同时还能够使用较少的主题进行控制生成。