Cross-modal feature extraction and integration have led to steady performance
improvements in few-shot learning tasks due to generating richer features.
However, existing multi-modal object detection (MM-OD) methods degrade when
facing significant domain-shift and are sample insufficient. We hypothesize
that rich text information could more effectively help the model to build a
knowledge relationship between the vision instance and its language description
and can help mitigate domain shift. Specifically, we study the Cross-Domain
few-shot generalization of MM-OD (CDMM-FSOD) and propose a meta-learning based
multi-modal few-shot object detection method that utilizes rich text semantic
information as an auxiliary modality to achieve domain adaptation in the
context of FSOD. Our proposed network contains (i) a multi-modal feature
aggregation module that aligns the vision and language support feature
embeddings and (ii) a rich text semantic rectify module that utilizes
bidirectional text feature generation to reinforce multi-modal feature
alignment and thus to enhance the model's language understanding capability. We
evaluate our model on common standard cross-domain object detection datasets
and demonstrate that our approach considerably outperforms existing FSOD
methods.

我们提出了一种多模态小样本物体检测方法，利用丰富的文本语义信息作为辅助模态来实现领域自适应，通过多模态特征聚合模块和丰富的文本语义校正模块增强模型的语言理解能力，实验证明该方法明显优于现有的小样本物体检测方法。