Transfer learning is a widely used method to build high performing computer
vision models. In this paper, we study the efficacy of transfer learning by
examining how the choice of data impacts performance. We find that more
pre-training data does not always help, and transfer performance depends on a
judicious choice of pre-training data. These findings are important given the
continued increase in dataset sizes. We further propose domain adaptive
transfer learning, a simple and effective pre-training method using importance
weights computed based on the target dataset. Our method to compute importance
weights follow from ideas in domain adaptation, and we show a novel application
to transfer learning. Our methods achieve state-of-the-art results on multiple
fine-grained classification datasets and are well-suited for use in practice.

本文研究了传递学习的有效性，并发现原先认为的更多的预训练数据不一定就能提高模型性能，而是需要审慎选择预训练数据。同时，作者提出了基于目标数据集计算重要性权重的领域自适应传递学习方法，并在多个细粒度分类数据集上取得了最先进的结果。