Language agents have shown some ability to interact with an external
environment, e.g., a virtual world such as ScienceWorld, to perform complex
tasks, e.g., growing a plant, without the startup costs of reinforcement
learning. However, despite their zero-shot capabilities, these agents to date
do not continually improve over time beyond performance refinement on a
specific task. Here we present CLIN, the first language-based agent to achieve
this, so that it continually improves over multiple trials, including when both
the environment and task are varied, and without requiring parameter updates.
Our approach is to use a persistent, dynamic, textual memory centered on causal
abstractions (rather than general "helpful hints") that is regularly updated
after each trial so that the agent gradually learns useful knowledge for new
trials. In the ScienceWorld benchmark, CLIN is able to continually improve on
repeated trials on the same task and environment, outperforming
state-of-the-art reflective language agents like Reflexion by 23 absolute
points. CLIN can also transfer its learning to new environments (or new tasks),
improving its zero-shot performance by 4 points (13 for new tasks) and can
further improve performance there through continual memory updates, enhancing
performance by an additional 17 points (7 for new tasks). This suggests a new
architecture for agents built on frozen models that can still continually and
rapidly improve over time.

CLIN 是第一个语言驱动的智能体，它通过持续更新的文本内存，不断改进表现并能够在变化的环境和任务中迁移学习，使得智能体的性能逐渐提升。

CLIN: 一个用于快速任务适应和泛化的持续学习语言代理

CLIN: A Continually Learning Language Agent for Rapid Task Adaptation  and Generalization

Huge pretrained language models (LMs) have demonstrated surprisingly good
zero-shot capabilities on a wide variety of tasks. This gives rise to the
appealing vision of a single, versatile model with a wide range of
functionalities across disparate applications. However, current leading
techniques for leveraging a "frozen" LM -- i.e., leaving its weights untouched
-- still often underperform fine-tuning approaches which modify these weights
in a task-dependent way. Those, in turn, suffer forgetfulness and compromise
versatility, suggesting a tradeoff between performance and versatility. The
main message of this paper is that current frozen-model techniques such as
prompt tuning are only the tip of the iceberg, and more powerful methods for
leveraging frozen LMs can do just as well as fine tuning in challenging domains
without sacrificing the underlying model's versatility. To demonstrate this, we
introduce three novel methods for leveraging frozen models: input-dependent
prompt tuning, frozen readers, and recursive LMs, each of which vastly improves
on current frozen-model approaches. Indeed, some of our methods even outperform
fine-tuning approaches in domains currently dominated by the latter. The
computational cost of each method is higher than that of existing frozen model
methods, but still negligible relative to a single pass through a huge frozen
LM. Each of these methods constitutes a meaningful contribution in its own
right, but by presenting these contributions together we aim to convince the
reader of a broader message that goes beyond the details of any given method:
that frozen models have untapped potential and that fine-tuning is often
unnecessary.

提出了三种新方法：依赖输入的提示调整、固定读者和递归 LM，这些方法广泛改进了当前的固定模型方法。虽然每种方法的计算成本比现有的固定模型方法高，但仍然可以忽略不计，而且这些方法证明了固定模型具有未开发的潜力，而精细调整往往是不必要的。