Traditional deep neural networks typically use end-to-end backpropagation,
which often places a big burden on GPU memory. Another promising training
method is local learning, which involves splitting the network into blocks and
training them in parallel with the help of an auxiliary network. Local learning
has been widely studied and applied to image classification tasks, and its
performance is comparable to that of end-to-end method. However, different
image tasks often rely on different feature representations, which is difficult
for typical auxiliary networks to adapt to. To solve this problem, we propose
the construction method of Global-Local Collaborative Auxiliary Network
(GLCAN), which provides a macroscopic design approach for auxiliary networks.
This is the first demonstration that local learning methods can be successfully
applied to other tasks such as object detection and super-resolution. GLCAN not
only saves a lot of GPU memory, but also has comparable performance to an
end-to-end approach on data sets for multiple different tasks.

传统的深度神经网络通常使用端到端的反向传播方法，在 GPU 内存上往往产生很大的负担。另一种有前途的训练方法是局部学习，它将网络分为多个块，并借助辅助网络并行训练它们。局部学习已被广泛研究和应用于图像分类任务中，并且其性能与端到端方法相当。然而，不同的图像任务通常依赖于不同的特征表示，这对于典型的辅助网络而言是困难的。为了解决这个问题，我们提出了全局 - 局部协作辅助网络（GLCAN）的构建方法，为辅助网络提供了宏观设计方法。这是局部学习方法首次成功应用于目标检测和超分辨率等其他任务的示范。GLCAN 不仅节省大量 GPU 内存，而且在多个不同任务的数据集上具有与端到端方法相当的性能。

GLCAN: 具有全局 - 本地协作辅助网络的本地学习

GLCAN: Global-Local Collaborative Auxiliary Network for Local Learning

In contrast to the natural capabilities of humans to learn new tasks in a
sequential fashion, neural networks are known to suffer from catastrophic
forgetting, where the model's performances on old tasks drop dramatically after
being optimized for a new task. Since then, the continual learning (CL)
community has proposed several solutions aiming to equip the neural network
with the ability to learn the current task (plasticity) while still achieving
high accuracy on the previous tasks (stability). Despite remarkable
improvements, the plasticity-stability trade-off is still far from being solved
and its underlying mechanism is poorly understood. In this work, we propose
Auxiliary Network Continual Learning (ANCL), a novel method that applies an
additional auxiliary network which promotes plasticity to the continually
learned model which mainly focuses on stability. More concretely, the proposed
framework materializes in a regularizer that naturally interpolates between
plasticity and stability, surpassing strong baselines on task incremental and
class incremental scenarios. Through extensive analyses on ANCL solutions, we
identify some essential principles beneath the stability-plasticity trade-off.

通过引入辅助网络实现稳定性与可塑性平衡的新方法 ANCL，在任务递增和类递增场景下表现优异，并揭示了稳定性与可塑性之间的基本原则。

通过辅助网络在连续学习中实现更好的稳定性 - 可塑性平衡

Achieving a Better Stability-Plasticity Trade-off via Auxiliary Networks  in Continual Learning

Streaming classification methods assume the number of input features is fixed
and always received. But in many real-world scenarios demand is some input
features are reliable while others are unreliable or inconsistent. In this
paper, we propose a novel deep learning-based model called Auxiliary Network
(Aux-Net), which is scalable and agile. It employs a weighted ensemble of
classifiers to give a final outcome. The Aux-Net model is based on the hedging
algorithm and online gradient descent. It employs a model of varying depth in
an online setting using single pass learning. Aux-Net is a foundational work
towards scalable neural network model for a dynamic complex environment
requiring ad hoc or inconsistent input data. The efficacy of Aux-Net is shown
on public dataset.

该研究提出了一种基于深度学习的模型 —— 辅助网络（Aux-Net），其采用加权集成分类器来处理输入特征数量的变化，具有可扩展性和敏捷性。Aux-Net 基于对冲算法和在线梯度下降算法，使用单次学习中的深度变化模型，适用于动态复杂环境，使能够处理不连贯的输入数据。在公共数据集上，实验证明了辅助网络的有效性。