Recently, DNN model compression based on network architecture design, e.g.,
SqueezeNet, attracted a lot attention. No accuracy drop on image classification
is observed on these extremely compact networks, compared to well-known models.
An emerging question, however, is whether these model compression techniques
hurt DNN's learning ability other than classifying images on a single dataset.
Our preliminary experiment shows that these compression methods could degrade
domain adaptation (DA) ability, though the classification performance is
preserved. Therefore, we propose a new compact network architecture and
unsupervised DA method in this paper. The DNN is built on a new basic module
Conv-M which provides more diverse feature extractors without significantly
increasing parameters. The unified framework of our DA method will
simultaneously learn invariance across domains, reduce divergence of feature
representations, and adapt label prediction. Our DNN has 4.1M parameters, which
is only 6.7% of AlexNet or 59% of GoogLeNet. Experiments show that our DNN
obtains GoogLeNet-level accuracy both on classification and DA, and our DA
method slightly outperforms previous competitive ones. Put all together, our DA
strategy based on our DNN achieves state-of-the-art on sixteen of total
eighteen DA tasks on popular Office-31 and Office-Caltech datasets.

本文介绍了一种基于新型模块 Conv-M 的超轻量级深度神经网络，该神经网络在不降低单数据集分类精度的前提下，提高了领域自适应 (DA) 任务的准确性，通过无监督的 DA 方法，在提高模型 invariance 同时，也适应了标签预测。该神经网络仅有 410 万参数，是 AlexNet 的 6.7％或 GoogLeNet 的 59％，并在常见的 Office-31 和 Office-Caltech 数据集上达到了 18 个 DA 任务中的 16 个的最新最佳水平。