Recent progress in large language models (LLMs) has demonstrated the ability
to learn and leverage Internet-scale knowledge through pre-training with
autoregressive models. Unfortunately, applying such models to settings with
embodied agents, such as robots, is challenging due to their lack of experience
with the physical world, inability to parse non-language observations, and
ignorance of rewards or safety constraints that robots may require. On the
other hand, language-conditioned robotic policies that learn from interaction
data can provide the necessary grounding that allows the agent to be correctly
situated in the real world, but such policies are limited by the lack of
high-level semantic understanding due to the limited breadth of the interaction
data available for training them. Thus, if we want to make use of the semantic
knowledge in a language model while still situating it in an embodied setting,
we must construct an action sequence that is both likely according to the
language model and also realizable according to grounded models of the
environment. We frame this as a problem similar to probabilistic filtering:
decode a sequence that both has high probability under the language model and
high probability under a set of grounded model objectives. We demonstrate this
guided decoding strategy is able to solve complex, long-horizon embodiment
tasks in a robotic setting by leveraging the knowledge of both models. The
project's website can be found at this http URL.

本论文提出了一种利用大型语言模型进行机器人交互的方法，用于解决理解语义知识和实现机器人任务之间的矛盾问题，其中引入了基于概率过滤的策略来采用语言模型和基于物理环境的模型两者的优势，通过指导解码策略可以实现复杂的机器人长视程任务。

基于接地模型指导机器人控制的文本生成

Grounded Decoding: Guiding Text Generation with Grounded Models for Robot Control

The end-to-end nature of neural machine translation (NMT) removes many ways
of manually guiding the translation process that were available in older
paradigms. Recent work, however, has introduced a new capability: lexically
constrained or guided decoding, a modification to beam search that forces the
inclusion of pre-specified words and phrases in the output. However, while
theoretically sound, existing approaches have computational complexities that
are either linear (Hokamp and Liu, 2017) or exponential (Anderson et al., 2017)
in the number of constraints. We present a algorithm for lexically constrained
decoding with a complexity of O(1) in the number of constraints. We demonstrate
the algorithms remarkable ability to properly place these constraints, and use
it to explore the shaky relationship between model and BLEU scores. Our
implementation is available as part of Sockeye.

该论文旨在介绍一种新的机器翻译解码算法 —— 词汇约束解码法，该算法能够快速而准确地在输出中包含预定的单词和短语，并探索模型与 BLEU 得分之间的关系，与此同时它还能够极大地优化计算复杂性。