Few-shot object detection (FSOD) benchmarks have advanced techniques for
detecting new categories with limited annotations. Existing benchmarks
repurpose well-established datasets like COCO by partitioning categories into
base and novel classes for pre-training and fine-tuning respectively. However,
these benchmarks do not reflect how FSOD is deployed in practice. Rather than
only pre-training on a small number of base categories, we argue that it is
more practical to fine-tune a foundation model (e.g., a vision-language model
(VLM) pre-trained on web-scale data) for a target domain. Surprisingly, we find
that zero-shot inference from VLMs like GroundingDINO significantly outperforms
the state-of-the-art (48.3 vs. 33.1 AP) on COCO. However, such zero-shot models
can still be misaligned to target concepts of interest. For example, trailers
on the web may be different from trailers in the context of autonomous
vehicles. In this work, we propose Foundational FSOD, a new benchmark protocol
that evaluates detectors pre-trained on any external datasets and fine-tuned on
K-shots per target class. Further, we note that current FSOD benchmarks are
actually federated datasets containing exhaustive annotations for each category
on a subset of the data. We leverage this insight to propose simple strategies
for fine-tuning VLMs with federated losses. We demonstrate the effectiveness of
our approach on LVIS and nuImages, improving over prior work by 5.9 AP.

本论文提出了一种基于少样本目标检测的新的评估协议，旨在在任何外部数据集上预训练和微调检测器，并且通过利用联合学习策略，改进了现有方法，并在 LVIS 和 nuImages 上获得 5.9 个 AP 的提升。