Recent advancements in large language models (LLMs) have shown potential for
human-like agents. To help these agents adapt to new tasks without extensive
human supervision, we propose the Learning through Communication (LTC)
paradigm, a novel training approach enabling LLM agents to improve continuously
through interactions with their environments and other agents. Recent
advancements in large language models (LLMs) have shown potential for
human-like agents. To help these agents adapt to new tasks without extensive
human supervision, we propose the Learning through Communication (LTC)
paradigm, a novel training approach enabling LLM agents to improve continuously
through interactions with their environments and other agents. Through
iterative exploration and PPO training, LTC empowers the agent to assimilate
short-term experiences into long-term memory. To optimize agent interactions
for task-specific learning, we introduce three structured communication
patterns: Monologue, Dialogue, and Analogue-tailored for common tasks such as
decision-making, knowledge-intensive reasoning, and numerical reasoning. We
evaluated LTC on three datasets: ALFWorld (decision-making), HotpotQA
(knowledge-intensive reasoning), and GSM8k (numerical reasoning). On ALFWorld,
it exceeds the instruction tuning baseline by 12% in success rate. On HotpotQA,
LTC surpasses the instruction-tuned LLaMA-7B agent by 5.1% in EM score, and it
outperforms the instruction-tuned 9x larger PaLM-62B agent by 0.6%. On GSM8k,
LTC outperforms the CoT-Tuning baseline by 3.6% in accuracy. The results
showcase the versatility and efficiency of the LTC approach across diverse
domains. We will open-source our code to promote further development of the
community.

基于大语言模型，提出了通过交流学习（LTC）的训练方法，使得代理人能够不断地通过与环境和其他代理的互动进行改进，通过三种结构化通信模式进行特定任务的学习，并在不同领域展现了 LTC 方法的多样性和效率。

通过通信改进 LLM 代理

Adapting LLM Agents Through Communication

Model-agnostic meta-learning (MAML) is one of the most popular and widely
adopted meta-learning algorithms, achieving remarkable success in various
learning problems. Yet, with the unique design of nested inner-loop and
outer-loop updates, which govern the task-specific and meta-model-centric
learning, respectively, the underlying learning objective of MAML remains
implicit and thus impedes a more straightforward understanding of it. In this
paper, we provide a new perspective of the working mechanism of MAML. We
discover that MAML is analogous to a meta-learner using a supervised
contrastive objective. The query features are pulled towards the support
features of the same class and against those of different classes. Such
contrastiveness is experimentally verified via an analysis based on the cosine
similarity. Moreover, we reveal that vanilla MAML has an undesirable
interference term originating from the random initialization and the cross-task
interaction. We thus propose a simple but effective technique, zeroing trick,
to alleviate the interference. Extensive experiments are conducted on both
mini-ImageNet and Omniglot datasets to demonstrate the consistent improvement
brought by our proposed method, validating its effectiveness.

本文提出了对 MAML 工作机制的新视角，将其类比为使用有监督对比目标的元学习器，并提出 zeroing trick 技术来缓解其干扰项，实验证明该技术提升了 MAML 在任务特定学习中的性能。

MAML 是分类任务中的噪声对比学习器

MAML is a Noisy Contrastive Learner in Classification

While tasks could come with varying the number of instances and classes in
realistic settings, the existing meta-learning approaches for few-shot
classification assume that the number of instances per task and class is fixed.
Due to such restriction, they learn to equally utilize the meta-knowledge
across all the tasks, even when the number of instances per task and class
largely varies. Moreover, they do not consider distributional difference in
unseen tasks, on which the meta-knowledge may have less usefulness depending on
the task relatedness. To overcome these limitations, we propose a novel
meta-learning model that adaptively balances the effect of the meta-learning
and task-specific learning within each task. Through the learning of the
balancing variables, we can decide whether to obtain a solution by relying on
the meta-knowledge or task-specific learning. We formulate this objective into
a Bayesian inference framework and tackle it using variational inference. We
validate our Bayesian Task-Adaptive Meta-Learning (Bayesian TAML) on multiple
realistic task- and class-imbalanced datasets, on which it significantly
outperforms existing meta-learning approaches. Further ablation study confirms
the effectiveness of each balancing component and the Bayesian learning
framework.

本文提出了一种新型元学习模型，在每个任务中自适应平衡元学习和任务特定学习的效果，并将其制定为贝叶斯推理框架，在多个现实的任务和课程不平衡的数据集上验证了其有效性。