Transfer of pre-trained representations can improve sample efficiency and
reduce computational requirements for new tasks. However, representations used
for transfer are usually generic, and are not tailored to a particular
distribution of downstream tasks. We explore the use of expert representations
for transfer with a simple, yet effective, strategy. We train a diverse set of
experts by exploiting existing label structures, and use cheap-to-compute
performance proxies to select the relevant expert for each target task. This
strategy scales the process of transferring to new tasks, since it does not
revisit the pre-training data during transfer. Accordingly, it requires little
extra compute per target task, and results in a speed-up of 2-3 orders of
magnitude compared to competing approaches. Further, we provide an
adapter-based architecture able to compress many experts into a single model.
We evaluate our approach on two different data sources and demonstrate that it
outperforms baselines on over 20 diverse vision tasks in both cases.

本文探讨了针对不同目标任务使用专家表示进行转移学习的有效策略，并提出了一种基于适配器的体系结构，能够将多个专家压缩到单个模型中，并在不同数据源上评估其性能，证明其在超过 20 种不同的视觉任务上优于基线方法。