Even after fine-tuning and reinforcement learning, large language models (LLMs) can be difficult, if not impossible, to control reliably with prompts alone. We propose a new inference-time approach to enforcing syntactic and semantic constraints on the outputs of LLMs, called sequential Monte Carlo (SMC) steering. The key idea is to specify language generation tasks as posterior inference problems in a class of discrete probabilistic sequence models, and replace standard decoding with sequential Monte Carlo inference. For a computational cost similar to that of beam search, SMC can steer LLMs to solve diverse tasks, including infilling, generation under syntactic constraints, and prompt intersection. To facilitate experimentation with SMC steering, we present a probabilistic programming library, LLaMPPL (https://github.com/probcomp/LLaMPPL), for concisely specifying new generation tasks as language model probabilistic programs, and automating steering of LLaMA-family Transformers.

提出了一种基于序列蒙特卡洛（SMC）引导的推理时方法，能够在后验推理问题的框架下对大型语言模型输出的句法和语义进行控制，包括缺失填充，句法约束下的生成和提示交集等任务，并提供了一个名为LLaMPPL的概率编程库以便于对LLaMA Transformers 进行处理。

利用概率编程对大语言模型进行顺序蒙特卡罗控制