Given an existing trained neural network, it is often desirable to learn new
capabilities without hindering performance of those already learned. Existing
approaches either learn sub-optimal solutions, require joint training, or incur
a substantial increment in the number of parameters for each added domain,
typically as many as the original network. We propose a method called
\emph{Deep Adaptation Networks} (DAN) that constrains newly learned filters to
be linear combinations of existing ones. DANs precisely preserve performance on
the original domain, require a fraction (typically 13\%, dependent on network
architecture) of the number of parameters compared to standard fine-tuning
procedures and converge in less cycles of training to a comparable or better
level of performance. When coupled with standard network quantization
techniques, we further reduce the parameter cost to around 3\% of the original
with negligible or no loss in accuracy. The learned architecture can be
controlled to switch between various learned representations, enabling a single
network to solve a task from multiple different domains. We conduct extensive
experiments showing the effectiveness of our method on a range of image
classification tasks and explore different aspects of its behavior.

提出了一种名为 Deep Adaptation Networks 的方法，通过在现有神经网络中添加线性组合的新滤波器来实现学习新技能，而不影响已经学习的性能，以及在各种图像分类任务中表现出良好的效果。