Knowledge amalgamation (KA) is a novel deep model reusing task aiming to
transfer knowledge from several well-trained teachers to a multi-talented and
compact student. Currently, most of these approaches are tailored for
convolutional neural networks (CNNs). However, there is a tendency that
transformers, with a completely different architecture, are starting to
challenge the domination of CNNs in many computer vision tasks. Nevertheless,
directly applying the previous KA methods to transformers leads to severe
performance degradation. In this work, we explore a more effective KA scheme
for transformer-based object detection models. Specifically, considering the
architecture characteristics of transformers, we propose to dissolve the KA
into two aspects: sequence-level amalgamation (SA) and task-level amalgamation
(TA). In particular, a hint is generated within the sequence-level amalgamation
by concatenating teacher sequences instead of redundantly aggregating them to a
fixed-size one as previous KA works. Besides, the student learns heterogeneous
detection tasks through soft targets with efficiency in the task-level
amalgamation. Extensive experiments on PASCAL VOC and COCO have unfolded that
the sequence-level amalgamation significantly boosts the performance of
students, while the previous methods impair the students. Moreover, the
transformer-based students excel in learning amalgamated knowledge, as they
have mastered heterogeneous detection tasks rapidly and achieved superior or at
least comparable performance to those of the teachers in their specializations.

本文探讨了基于 Transformer 的物体检测模型用于知识汇集的更有效的方案，提出了序列级别和任务级别两个方面的知识汇集，其中利用教师序列拼接生成提示进行序列级别的知识汇集，而利用软目标进行任务级别的知识汇集，实现了快速学习多样化检测任务并达到或超越教师的性能。