Multi-task networks are commonly utilized to alleviate the need for a large
number of highly specialized single-task networks. However, two common
challenges in developing multi-task models are often overlooked in literature.
First, enabling the model to be inherently incremental, continuously
incorporating information from new tasks without forgetting the previously
learned ones (incremental learning). Second, eliminating adverse interactions
amongst tasks, which has been shown to significantly degrade the single-task
performance in a multi-task setup (task interference). In this paper, we show
that both can be achieved simply by reparameterizing the convolutions of
standard neural network architectures into a non-trainable shared part (filter
bank) and task-specific parts (modulators), where each modulator has a fraction
of the filter bank parameters. Thus, our reparameterization enables the model
to learn new tasks without adversely affecting the performance of existing
ones. The results of our ablation study attest the efficacy of the proposed
reparameterization. Moreover, our method achieves state-of-the-art on two
challenging multi-task learning benchmarks, PASCAL-Context and NYUD, and also
demonstrates superior incremental learning capability as compared to its close
competitors.

该研究提出了一种新的重新参数化方法，通过将标准神经网络体系结构的卷积重新参数化成一个不可训练的共享部分（滤波器银行）和任务特定部分（调制器），从而使模型能够学习新任务而不会对现有任务的性能产生负面影响。该方法在 PASCAL-Context 和 NYUD 两个任务上达到了最先进的表现，并且证明了其比竞争对手具有更优秀的增量学习能力。