This work presents a method for adding multiple tasks to a single, fixed deep neural network without affecting performance on already learned tasks. By building upon concepts from network quantization and sparsification, we learn binary masks that "piggyback", or are applied to an existing network to provide good performance on a new task. These masks are learned in an end-to-end differentiable fashion, and incur a low overhead of 1 bit per network parameter, per task. Even though the underlying network is fixed, the ability to mask certain weights allows for the learning of a large number of filters. We show improved performance on a variety of classification tasks, including those with large domain shifts from the natural images of ImageNet. Unlike prior work, we can augment the capabilities of a network without suffering from catastrophic forgetting or competition between tasks, while incurring the least overhead per added task. We demonstrate the applicability of our method to multiple architectures, and obtain accuracies comparable with individual networks trained per task.

本文提出了一种方法，通过二进制掩码的方式嫁接在已有的神经网络上，从而实现在新任务上达到较好性能的同时不影响旧任务的性能，该方法基于网络量化和剪枝的思想，以端对端可微的方式学习掩码，能够像微调网络一样学习大量的滤波器，适用于大规模分类，性能稳定、无遗忘现像，且任务顺序不影响性能。

Piggyback: 学习掩蔽权重以将单一网络适应于多个任务