A massive number of well-trained deep networks have been released by developers online. These networks may focus on different tasks and in many cases are optimized for different datasets. In this paper, we study how to exploit such heterogeneous pre-trained networks, known as teachers, so as to train a customized student network that tackles a set of selective tasks defined by the user. We assume no human annotations are available, and each teacher may be either single- or multi-task. To this end, we introduce a dual-step strategy that first extracts the task-specific knowledge from the heterogeneous teachers sharing the same sub-task, and then amalgamates the extracted knowledge to build the student network. To facilitate the training, we employ a selective learning scheme where, for each unlabelled sample, the student learns adaptively from only the teacher with the least prediction ambiguity. We evaluate the proposed approach on several datasets and experimental results demonstrate that the student, learned by such adaptive knowledge amalgamation, achieves performances even better than those of the teachers.

本篇文章探讨了如何利用预先训练好的深度网络来训练一个个性化的学生网络，以解决用户选择的任务，文章提出了一种双步骤策略，其中第一步从具有相同子任务的异构教师中提取任务特定知识，然后将提取的知识混合，来构建学生网络，以自我的方式从带有最小预测模糊性的教师那里学习，实验结果表明基于此种自适应知识归纳的学生网络的性能甚至优于教师。

通过自适应知识融合从异质教师中定制学生网络