Both entropy-minimizing and entropy-maximizing (curiosity) objectives for
unsupervised reinforcement learning (RL) have been shown to be effective in
different environments, depending on the environment's level of natural
entropy. However, neither method alone results in an agent that will
consistently learn intelligent behavior across environments. In an effort to
find a single entropy-based method that will encourage emergent behaviors in
any environment, we propose an agent that can adapt its objective online,
depending on the entropy conditions by framing the choice as a multi-armed
bandit problem. We devise a novel intrinsic feedback signal for the bandit,
which captures the agent's ability to control the entropy in its environment.
We demonstrate that such agents can learn to control entropy and exhibit
emergent behaviors in both high- and low-entropy regimes and can learn skillful
behaviors in benchmark tasks. Videos of the trained agents and summarized
findings can be found on our project page
this https URL

提出了一个基于多臂赌博机问题的代理模型，该模型根据环境的熵条件动态调整目标，从而鼓励在不同熵环境中出现新的行为和学习技能。

自适应惊喜内在动机的无监督强化学习

Surprise-Adaptive Intrinsic Motivation for Unsupervised Reinforcement  Learning

Recent advancements in large language models (LLMs) have shown potential for
human-like agents. To help these agents adapt to new tasks without extensive
human supervision, we propose the Learning through Communication (LTC)
paradigm, a novel training approach enabling LLM agents to improve continuously
through interactions with their environments and other agents. Recent
advancements in large language models (LLMs) have shown potential for
human-like agents. To help these agents adapt to new tasks without extensive
human supervision, we propose the Learning through Communication (LTC)
paradigm, a novel training approach enabling LLM agents to improve continuously
through interactions with their environments and other agents. Through
iterative exploration and PPO training, LTC empowers the agent to assimilate
short-term experiences into long-term memory. To optimize agent interactions
for task-specific learning, we introduce three structured communication
patterns: Monologue, Dialogue, and Analogue-tailored for common tasks such as
decision-making, knowledge-intensive reasoning, and numerical reasoning. We
evaluated LTC on three datasets: ALFWorld (decision-making), HotpotQA
(knowledge-intensive reasoning), and GSM8k (numerical reasoning). On ALFWorld,
it exceeds the instruction tuning baseline by 12% in success rate. On HotpotQA,
LTC surpasses the instruction-tuned LLaMA-7B agent by 5.1% in EM score, and it
outperforms the instruction-tuned 9x larger PaLM-62B agent by 0.6%. On GSM8k,
LTC outperforms the CoT-Tuning baseline by 3.6% in accuracy. The results
showcase the versatility and efficiency of the LTC approach across diverse
domains. We will open-source our code to promote further development of the
community.

基于大语言模型，提出了通过交流学习（LTC）的训练方法，使得代理人能够不断地通过与环境和其他代理的互动进行改进，通过三种结构化通信模式进行特定任务的学习，并在不同领域展现了 LTC 方法的多样性和效率。