Designing generalizable agents capable of adapting to diverse embodiments has
achieved significant attention in Reinforcement Learning (RL), which is
critical for deploying RL agents in various real-world applications. Previous
Cross-Embodiment RL approaches have focused on transferring knowledge across
embodiments within specific tasks. These methods often result in knowledge
tightly coupled with those tasks and fail to adequately capture the distinct
characteristics of different embodiments. To address this limitation, we
introduce the notion of Cross-Embodiment Unsupervised RL (CEURL), which
leverages unsupervised learning to enable agents to acquire embodiment-aware
and task-agnostic knowledge through online interactions within reward-free
environments. We formulate CEURL as a novel Controlled Embodiment Markov
Decision Process (CE-MDP) and systematically analyze CEURL's pre-training
objectives under CE-MDP. Based on these analyses, we develop a novel algorithm
Pre-trained Embodiment-Aware Control (PEAC) for handling CEURL, incorporating
an intrinsic reward function specifically designed for cross-embodiment
pre-training. PEAC not only provides an intuitive optimization strategy for
cross-embodiment pre-training but also can integrate flexibly with existing
unsupervised RL methods, facilitating cross-embodiment exploration and skill
discovery. Extensive experiments in both simulated (e.g., DMC and Robosuite)
and real-world environments (e.g., legged locomotion) demonstrate that PEAC
significantly improves adaptation performance and cross-embodiment
generalization, demonstrating its effectiveness in overcoming the unique
challenges of CEURL.

通过无监督学习以及预先训练的方式，提出了一种适用于不同体现形式的强化学习的控制策略，能够显著提高适应性能和跨体现形式的泛化能力。

PEAC：无监督预训练跨体现感知强化学习

PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement  Learning

Adapting large language models (LLMs) to unseen tasks with in-context
training samples without fine-tuning remains an important research problem. To
learn a robust LLM that adapts well to unseen tasks, multiple meta-training
approaches have been proposed such as MetaICL and MetaICT, which involve
meta-training pre-trained LLMs on a wide variety of diverse tasks. These
meta-training approaches essentially perform in-context multi-task fine-tuning
and evaluate on a disjointed test set of tasks. Even though they achieve
impressive performance, their goal is never to compute a truly general set of
parameters. In this paper, we propose MAML-en-LLM, a novel method for
meta-training LLMs, which can learn truly generalizable parameters that not
only perform well on disjointed tasks but also adapts to unseen tasks. We see
an average increase of 2% on unseen domains in the performance while a massive
4% improvement on adaptation performance. Furthermore, we demonstrate that
MAML-en-LLM outperforms baselines in settings with limited amount of training
data on both seen and unseen domains by an average of 2%. Finally, we discuss
the effects of type of tasks, optimizers and task complexity, an avenue barely
explored in meta-training literature. Exhaustive experiments across 7 task
settings along with two data settings demonstrate that models trained with
MAML-en-LLM outperform SOTA meta-training approaches.

我们提出了 MAML-en-LLM，一种新的元训练大型语言模型的方法，可以学习到真正可泛化的参数，不仅在不同任务上表现良好，还适应了未知任务，并且在性能和适应性能上得到了显著提升。

MAML-en-LLM：模型不可知的元训练 LLMs 对于改进上下文学习

MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved  In-Context Learning

Fully test-time adaptation aims to adapt the network model based on
sequential analysis of input samples during the inference stage to address the
cross-domain performance degradation problem of deep neural networks. We take
inspiration from the biological plausibility learning where the neuron
responses are tuned based on a local synapse-change procedure and activated by
competitive lateral inhibition rules. Based on these feed-forward learning
rules, we design a soft Hebbian learning process which provides an unsupervised
and effective mechanism for online adaptation. We observe that the performance
of this feed-forward Hebbian learning for fully test-time adaptation can be
significantly improved by incorporating a feedback neuro-modulation layer. It
is able to fine-tune the neuron responses based on the external feedback
generated by the error back-propagation from the top inference layers. This
leads to our proposed neuro-modulated Hebbian learning (NHL) method for fully
test-time adaptation. With the unsupervised feed-forward soft Hebbian learning
being combined with a learned neuro-modulator to capture feedback from external
responses, the source model can be effectively adapted during the testing
process. Experimental results on benchmark datasets demonstrate that our
proposed method can significantly improve the adaptation performance of network
models and outperforms existing state-of-the-art methods.

本文研究了深度神经网络领域中跨域性能退化问题，提出了一种基于完全测试时间自适应方法来提高网络适应性能的方法，其中采用了软式 Hebbian 学习过程和反馈神经调节层来辅助在线适应。实验结果表明，该方法可以显著改善网络模型的适应性能并超越现有的最新方法。

全测试时间适应的神经调制 Hebbian 学习

Neuro-Modulated Hebbian Learning for Fully Test-Time Adaptation

We aim at the problem named One-Shot Unsupervised Domain Adaptation. Unlike
traditional Unsupervised Domain Adaptation, it assumes that only one unlabeled
target sample can be available when learning to adapt. This setting is
realistic but more challenging, in which conventional adaptation approaches are
prone to failure due to the scarce of unlabeled target data. To this end, we
propose a novel Adversarial Style Mining approach, which combines the style
transfer module and task-specific module into an adversarial manner.
Specifically, the style transfer module iteratively searches for harder
stylized images around the one-shot target sample according to the current
learning state, leading the task model to explore the potential styles that are
difficult to solve in the almost unseen target domain, thus boosting the
adaptation performance in a data-scarce scenario. The adversarial learning
framework makes the style transfer module and task-specific module benefit each
other during the competition. Extensive experiments on both cross-domain
classification and segmentation benchmarks verify that ASM achieves
state-of-the-art adaptation performance under the challenging one-shot setting.

本文提出了一种新的对抗风格挖掘方法，将样式转移模块和任务特定模块相结合，通过对抗学习的方式解决一次样本不足的情况下的领域自适应问题，进而提高了领域自适应性能。