Large-vocabulary object detectors (LVDs) aim to detect objects of many
categories, which learn super objectness features and can locate objects
accurately while applied to various downstream data. However, LVDs often
struggle in recognizing the located objects due to domain discrepancy in data
distribution and object vocabulary. At the other end, recent vision-language
foundation models such as CLIP demonstrate superior open-vocabulary recognition
capability. This paper presents KGD, a Knowledge Graph Distillation technique
that exploits the implicit knowledge graphs (KG) in CLIP for effectively
adapting LVDs to various downstream domains. KGD consists of two consecutive
stages: 1) KG extraction that employs CLIP to encode downstream domain data as
nodes and their feature distances as edges, constructing KG that inherits the
rich semantic relations in CLIP explicitly; and 2) KG encapsulation that
transfers the extracted KG into LVDs to enable accurate cross-domain object
classification. In addition, KGD can extract both visual and textual KG
independently, providing complementary vision and language knowledge for object
localization and object classification in detection tasks over various
downstream domains. Experiments over multiple widely adopted detection
benchmarks show that KGD outperforms the state-of-the-art consistently by large
margins.

本文介绍了 KGD（Knowledge Graph Distillation）技术，它利用了 CLIP 中的隐式知识图，通过两个连续阶段将知识图传递到大词汇物体检测器（LVDs），从而在各种下游领域中实现准确的跨领域物体分类。实验证明，KGD 在多个广泛采用的检测基准上始终优于最先进水平。