Existing prompt-tuning methods have demonstrated impressive performances in
continual learning (CL), by selecting and updating relevant prompts in the
vision-transformer models. On the contrary, this paper aims to learn each task
by tuning the prompts in the direction orthogonal to the subspace spanned by
previous tasks' features, so as to ensure no interference on tasks that have
been learned to overcome catastrophic forgetting in CL. However, different from
the orthogonal projection in the traditional CNN architecture, the prompt
gradient orthogonal projection in the ViT architecture shows completely
different and greater challenges, i.e., 1) the high-order and non-linear
self-attention operation; 2) the drift of prompt distribution brought by the
LayerNorm in the transformer block. Theoretically, we have finally deduced two
consistency conditions to achieve the prompt gradient orthogonal projection,
which provide a theoretical guarantee of eliminating interference on previously
learned knowledge via the self-attention mechanism in visual prompt tuning. In
practice, an effective null-space-based approximation solution has been
proposed to implement the prompt gradient orthogonal projection. Extensive
experimental results demonstrate the effectiveness of anti-forgetting on four
class-incremental benchmarks with diverse pre-trained baseline models, and our
approach achieves superior performances to state-of-the-art methods. Our code
is available in the supplemental material.

通过对以前任务特征生成的子空间正交方向上调整提示信息，本文提出了在视觉提示调整中实现提示梯度正交投影的一种有效的空间近似解决方案。实验证明了该方法在四个增量学习基准测试中对防止遗忘的有效性，并且优于现有方法的表现。

零空间中的视觉指示调节对于连续学习的影响

Visual Prompt Tuning in Null Space for Continual Learning

Due to the comprehensive nature of this paper, it has been updated and split
into two separate papers: "A Framework For Benchmarking
Class-out-of-distribution Detection And Its Application To ImageNet" and "What
Can We Learn From The Selective Prediction And Uncertainty Estimation
Performance Of 523 Imagenet Classifiers". We recommend reading them instead.
Deep neural networks must be equipped with an uncertainty estimation
mechanism when deployed for risk-sensitive tasks. This paper studies the
relationship between deep architectures and their training regimes with their
corresponding selective prediction and uncertainty estimation performance. We
consider both in-distribution uncertainties and class-out-of-distribution ones.
Moreover, we consider some of the most popular estimation performance metrics
previously proposed including AUROC, ECE, AURC, and coverage for selective
accuracy constraint. We present a novel and comprehensive study of selective
prediction and the uncertainty estimation performance of 484 existing
pretrained deep ImageNet classifiers that are available at popular
repositories. We identify numerous and previously unknown factors that affect
uncertainty estimation and examine the relationships between the different
metrics. We find that distillation-based training regimes consistently yield
better uncertainty estimations than other training schemes such as vanilla
training, pretraining on a larger dataset and adversarial training. We also
provide strong empirical evidence showing that ViT is by far the most superior
architecture in terms of uncertainty estimation performance, judging by any
aspect, in both in-distribution and class-out-of-distribution scenarios.

探讨深度神经网络架构与训练机制与其相应的选择性预测和不确定性估计性能的关系，并在 484 个预训练的深度 Imagenet 分类器中进行了全面的选择性预测和不确定性估计性能研究，发现 ViT 架构在不确定性估计性能方面表现最优。