Continual learning with neural networks is an important learning framework in
AI that aims to learn a sequence of tasks well. However, it is often confronted
with three challenges: (1) overcome the catastrophic forgetting problem, (2)
adapt the current network to new tasks, and meanwhile (3) control its model
complexity. To reach these goals, we propose a novel approach named as
Continual Learning with Efficient Architecture Search, or CLEAS in short. CLEAS
works closely with neural architecture search (NAS) which leverages
reinforcement learning techniques to search for the best neural architecture
that fits a new task. In particular, we design a neuron-level NAS controller
that decides which old neurons from previous tasks should be reused (knowledge
transfer), and which new neurons should be added (to learn new knowledge). Such
a fine-grained controller allows one to find a very concise architecture that
can fit each new task well. Meanwhile, since we do not alter the weights of the
reused neurons, we perfectly memorize the knowledge learned from previous
tasks. We evaluate CLEAS on numerous sequential classification tasks, and the
results demonstrate that CLEAS outperforms other state-of-the-art alternative
methods, achieving higher classification accuracy while using simpler neural
architectures.

提出了一种名为 CLEAS 的新方法，通过与神经结构搜索（NAS）紧密协作，旨在解决神经网络连续学习所面临的挑战：克服灾难性遗忘问题，适应新任务，同时控制其模型复杂度，并通过在神经元层次上设计 NAS 控制器来加强知识转移，以实现更高的分类准确度和更简单的神经网络结构。

高效的连续学习架构搜索

Efficient Architecture Search for Continual Learning

An effective person re-identification (re-ID) model should learn feature
representations that are both discriminative, for distinguishing
similar-looking people, and generalisable, for deployment across datasets
without any adaptation. In this paper, we develop novel CNN architectures to
address both challenges. First, we present a re-ID CNN termed omni-scale
network (OSNet) to learn features that not only capture different spatial
scales but also encapsulate a synergistic combination of multiple scales,
namely omni-scale features. The basic building block consists of multiple
convolutional streams, each detecting features at a certain scale. For
omni-scale feature learning, a unified aggregation gate is introduced to
dynamically fuse multi-scale features with channel-wise weights. OSNet is
lightweight as its building blocks comprise factorised convolutions. Second, to
improve generalisable feature learning, we introduce instance normalisation
(IN) layers into OSNet to cope with cross-dataset discrepancies. Further, to
determine the optimal placements of these IN layers in the architecture, we
formulate an efficient differentiable architecture search algorithm. Extensive
experiments show that, in the conventional same-dataset setting, OSNet achieves
state-of-the-art performance, despite being much smaller than existing re-ID
models. In the more challenging yet practical cross-dataset setting, OSNet
beats most recent unsupervised domain adaptation methods without using any
target data. Our code and models are released at
\texttt{this https URL}.

本文提出了使用人物重新识别 CNN 网络建模的新方法，使用 multi-scale 特征和统一聚合门，还使用 instance normalization 处理跨数据集的不一致性，并提出了一种有效的可微架构搜索算法。通过实验验证，该方法在同数据集下表现良好，在不使用目标数据的情况下，跨数据集的表现甚至超过大多数最新的无监督域自适应方法。