Learning a universal policy across different robot morphologies can
significantly improve learning efficiency and enable zero-shot generalization
to unseen morphologies. However, learning a highly performant universal policy
requires sophisticated architectures like transformers (TF) that have larger
memory and computational cost than simpler multi-layer perceptrons (MLP). To
achieve both good performance like TF and high efficiency like MLP at inference
time, we propose HyperDistill, which consists of: (1) A morphology-conditioned
hypernetwork (HN) that generates robot-wise MLP policies, and (2) A policy
distillation approach that is essential for successful training. We show that
on UNIMAL, a benchmark with hundreds of diverse morphologies, HyperDistill
performs as well as a universal TF teacher policy on both training and unseen
test robots, but reduces model size by 6-14 times, and computational cost by
67-160 times in different environments. Our analysis attributes the efficiency
advantage of HyperDistill at inference time to knowledge decoupling, i.e., the
ability to decouple inter-task and intra-task knowledge, a general principle
that could also be applied to improve inference efficiency in other domains.

通过 HyperDistill 方法，我们能够在不同机器人形态之间学习一种通用策略，从而显著提高学习效率，并且实现对未见过形态的机器人的零 - shot 泛化，同时保持了与 transformers 相似的性能和与 multi-layer perceptrons 相似的高效性。

用于高效普适形态控制的形态条件化超网络蒸馏

Distilling Morphology-Conditioned Hypernetworks for Efficient Universal  Morphology Control

Learning a universal policy across different robot morphologies can
significantly improve learning efficiency and generalization in continuous
control. However, it poses a challenging multi-task reinforcement learning
problem, as the optimal policy may be quite different across robots and
critically depend on the morphology. Existing methods utilize graph neural
networks or transformers to handle heterogeneous state and action spaces across
different morphologies, but pay little attention to the dependency of a robot's
control policy on its morphology context. In this paper, we propose a
hierarchical architecture to better model this dependency via contextual
modulation, which includes two key submodules: (1) Instead of enforcing hard
parameter sharing across robots, we use hypernetworks to generate
morphology-dependent control parameters; (2) We propose a morphology-dependent
attention mechanism to modulate the interactions between different limbs in a
robot. Experimental results show that our method not only improves learning
performance on a diverse set of training robots, but also generalizes better to
unseen morphologies in a zero-shot fashion.

该论文提出了一种分层架构，通过上下文调节来更好地模拟机器人控制策略对其形态上下文的依赖关系，其中包括两个关键子模块：（1）使用超网络生成依赖形态的控制参数；（2）提出了一种形态依赖性的注意机制来调节机器人不同肢体之间的交互。实验结果表明，该方法不仅改善了各种训练机器人的学习性能，还以零样本方式更好地推广到未见过的形态。

通过上下文调节实现通用形态控制

Universal Morphology Control via Contextual Modulation

The discovery of complex multicellular organism development took millions of
years of evolution. The genome of such a multicellular organism guides the
development of its body from a single cell, including its control system. Our
goal is to imitate this natural process using a single neural cellular
automaton (NCA) as a genome for modular robotic agents. In the introduced
approach, called Neural Cellular Robot Substrate (NCRS), a single NCA guides
the growth of a robot and the cellular activity which controls the robot during
deployment. We also introduce three benchmark environments, which test the
ability of the approach to grow different robot morphologies. In this paper,
NCRSs are trained with covariance matrix adaptation evolution strategy
(CMA-ES), and covariance matrix adaptation MAP-Elites (CMA-ME) for quality
diversity, which we show leads to more diverse robot morphologies with higher
fitness scores. While the NCRS can solve the easier tasks from our benchmark
environments, the success rate reduces when the difficulty of the task
increases. We discuss directions for future work that may facilitate the use of
the NCRS approach for more complex domains.

通过引入神经细胞机器人基质（NCRS）的方法，使用单个神经细胞自动机（NCA）作为模块化机器人代理的基因组，使用协方差矩阵适应进化策略（CMA-ES）和协方差矩阵适应 MAP-Elites（CMA-ME）来训练 NCRS，以实现对机器人形态的成长和控制的模拟。研究表明，该方法在较简单的任务中表现良好，但在任务难度增加时，其成功率会降低。