Leveraging large-scale data can introduce performance gains on many computer vision tasks. Unfortunately, this does not happen in object detection when training a single model under multiple datasets together. We observe two main obstacles: taxonomy difference and bounding box annotation inconsistency, which introduces domain gaps in different datasets that prevents us from joint training. In this paper, we show that these two challenges can be effectively addressed by simply adapting object queries on language embedding of categories per dataset. We design a detection hub to dynamically adapt queries on category embedding based on the different distributions of datasets. Unlike previous methods attempted to learn a joint embedding for all datasets, our adaptation method can utilize the language embedding as semantic centers for common categories, while learning the semantic bias towards specific categories belonging to different datasets to handle annotation differences and make up the domain gaps. These novel improvements enable us to end-to-end train a single detector on multiple datasets simultaneously to fully take their advantages. Further experiments on joint training on multiple datasets demonstrate the significant performance gains over separate individual fine-tuned detectors.

本研究提出一种新的设计（称为 Detection Hub），它是数据集感知并具有类别对齐功能，通过学习一种数据集嵌入来自适应对象查询以及检测头中的卷积核，从而既减轻了对目标检测的数据集不一致性，又提供了一致的指导，以便检测器跨多个数据集学习。实验表明，检测中心进一步在各种数据集的UODB基准测试中实现了SoTA性能。

Detection Hub: 基于语言嵌入的查询适应统一物体检测数据集