Continual learning endeavors to equip the model with the capability to
integrate current task knowledge while mitigating the forgetting of past task
knowledge. Inspired by prompt tuning, prompt-based methods maintain a frozen
backbone and train with slight learnable prompts to minimize the catastrophic
forgetting that arises due to updating a large number of backbone parameters.
Nonetheless, these learnable prompts tend to concentrate on the discriminatory
knowledge of the current task while ignoring past task knowledge, leading to
that learnable prompts still suffering from catastrophic forgetting. This paper
introduces a novel rehearsal-free paradigm for continual learning termed
Hierarchical Prompts (H-Prompts), comprising three categories of prompts --
class prompt, task prompt, and general prompt. To effectively depict the
knowledge of past classes, class prompt leverages Bayesian Distribution
Alignment to model the distribution of classes in each task. To reduce the
forgetting of past task knowledge, task prompt employs Cross-task Knowledge
Excavation to amalgamate the knowledge encapsulated in the learned class
prompts of past tasks and current task knowledge. Furthermore, general prompt
utilizes Generalized Knowledge Exploration to deduce highly generalized
knowledge in a self-supervised manner. Evaluations on two benchmarks
substantiate the efficacy of the proposed H-Prompts, exemplified by an average
accuracy of 87.8% in Split CIFAR-100 and 70.6% in Split ImageNet-R.

本文介绍了一种无需复盘的延续学习新范式，称为分层提示（H-Prompts），包括三类提示 - 类提示、任务提示和通用提示。通过贝叶斯分布对齐来刻画过去类别的知识，通过跨任务知识挖掘来减少过去任务知识的遗忘，并利用自监督方式推导出高度概括的通用知识。在两个基准测试中，证明了所提出的 H-Prompts 的高效性，分别在 Split CIFAR-100 上达到了 87.8％的平均准确率，以及在 Split ImageNet-R 上达到了 70.6％的准确率。

无需重复训练的层次提示连续学习

Hierarchical Prompts for Rehearsal-free Continual Learning

Large-scale pre-trained models have demonstrated impressive performance in
vision and language tasks within open-world scenarios. Due to the lack of
comparable pre-trained models for 3D shapes, recent methods utilize
language-image pre-training to realize zero-shot 3D shape recognition. However,
due to the modality gap, pretrained language-image models are not confident
enough in the generalization to 3D shape recognition. Consequently, this paper
aims to improve the confidence with view selection and hierarchical prompts.
Leveraging the CLIP model as an example, we employ view selection on the vision
side by identifying views with high prediction confidence from multiple
rendered views of a 3D shape. On the textual side, the strategy of hierarchical
prompts is proposed for the first time. The first layer prompts several
classification candidates with traditional class-level descriptions, while the
second layer refines the prediction based on function-level descriptions or
further distinctions between the candidates. Remarkably, without the need for
additional training, our proposed method achieves impressive zero-shot 3D
classification accuracies of 84.44\%, 91.51\%, and 66.17\% on ModelNet40,
ModelNet10, and ShapeNet Core55, respectively. Furthermore, we will make the
code publicly available to facilitate reproducibility and further research in
this area.

通过视角选择和分层提示的策略，本研究旨在改进预训练模型在零样本三维形状识别中的信心，实现无需额外训练的令人印象深刻的分类准确性。

MV-CLIP：多视角 CLIP 用于零样本 3D 形状识别

MV-CLIP: Multi-View CLIP for Zero-shot 3D Shape Recognition

Large language models (LLMs) have demonstrated remarkable capabilities in
performing a range of instruction following tasks in few and zero-shot
settings. However, teaching LLMs to perform tasks on the web presents
fundamental challenges -- combinatorially large open-world tasks and variations
across web interfaces. We tackle these challenges by leveraging LLMs to
decompose web tasks into a collection of sub-tasks, each of which can be solved
by a low-level, closed-loop policy. These policies constitute a shared grammar
across tasks, i.e., new web tasks can be expressed as a composition of these
policies. We propose a novel framework, Hierarchical Policies for Web Actions
using LLMs (HeaP), that learns a set of hierarchical LLM prompts from
demonstrations for planning high-level tasks and executing them via a sequence
of low-level policies. We evaluate HeaP against a range of baselines on a suite
of web tasks, including MiniWoB++, WebArena, a mock airline CRM, as well as
live website interactions, and show that it is able to outperform prior works
using orders of magnitude less data.

大型语言模型（LLMs）通过使用层次化促进和学习低级策略，能够解决 Web 任务的组合和变化问题，从而在使用更少数据的情况下提高任务执行性能。