Continual reinforcement learning (CRL) empowers RL agents with the ability to
learn from a sequence of tasks, preserving previous knowledge and leveraging it
to facilitate future learning. However, existing methods often focus on
transferring low-level knowledge across similar tasks, which neglects the
hierarchical structure of human cognitive control, resulting in insufficient
knowledge transfer across diverse tasks. To enhance high-level knowledge
transfer, we propose a novel framework named Hi-Core (Hierarchical knowledge
transfer for Continual reinforcement learning), which is structured in two
layers: 1) the high-level policy formulation which utilizes the powerful
reasoning ability of the Large Language Model (LLM) to set goals and 2) the
low-level policy learning through RL which is oriented by high-level goals.
Moreover, the knowledge base (policy library) is constructed to store policies
that can be retrieved for hierarchical knowledge transfer. Experiments
conducted in MiniGrid have demonstrated the effectiveness of Hi-Core in
handling diverse CRL tasks, outperforming popular baselines.

Hi-Core 是一个新颖框架，用于在连续强化学习中增强高层次的知识传递，并通过大型语言模型（LLM）进行目标设置和低层次策略学习，并通过知识库存储策略实现层次化知识传递，实验证明其在处理多样化 CRL 任务上的有效性优于常见的基线模型。

Hi-Core：层次化知识迁移用于连续性强化学习

Hi-Core: Hierarchical Knowledge Transfer for Continual Reinforcement  Learning

Hierarchical multi-granularity classification (HMC) assigns hierarchical
multi-granularity labels to each object and focuses on encoding the label
hierarchy, e.g., ["Albatross", "Laysan Albatross"] from coarse-to-fine levels.
However, the definition of what is fine-grained is subjective, and the image
quality may affect the identification. Thus, samples could be observed at any
level of the hierarchy, e.g., ["Albatross"] or ["Albatross", "Laysan
Albatross"], and examples discerned at coarse categories are often neglected in
the conventional setting of HMC. In this paper, we study the HMC problem in
which objects are labeled at any level of the hierarchy. The essential designs
of the proposed method are derived from two motivations: (1) learning with
objects labeled at various levels should transfer hierarchical knowledge
between levels; (2) lower-level classes should inherit attributes related to
upper-level superclasses. The proposed combinatorial loss maximizes the
marginal probability of the observed ground truth label by aggregating
information from related labels defined in the tree hierarchy. If the observed
label is at the leaf level, the combinatorial loss further imposes the
multi-class cross-entropy loss to increase the weight of fine-grained
classification loss. Considering the hierarchical feature interaction, we
propose a hierarchical residual network (HRN), in which granularity-specific
features from parent levels acting as residual connections are added to
features of children levels. Experiments on three commonly used datasets
demonstrate the effectiveness of our approach compared to the state-of-the-art
HMC approaches and fine-grained visual classification (FGVC) methods exploiting
the label hierarchy.

该论文研究了层次多粒度分类问题，基于所提出的组合损失以及继承父级别特征的方法，设计了一种基于层次残差网络的深度神经网络结构，展示了该方法有效性在通常数据集上优于目前最先进的方法及精细分类方法利用标签层次结构。