Recent years have seen object detection robotic systems deployed in several
personal devices (e.g., home robots and appliances). This has highlighted a
challenge in their design, i.e., they cannot efficiently update their knowledge
to distinguish between general classes and user-specific instances (e.g., a dog
vs. user's dog). We refer to this challenging task as Instance-level
Personalized Object Detection (IPOD). The personalization task requires many
samples for model tuning and optimization in a centralized server, raising
privacy concerns. An alternative is provided by approaches based on recent
large-scale Foundation Models, but their compute costs preclude on-device
applications.
In our work we tackle both problems at the same time, designing a Few-Shot
IPOD strategy called AuXFT. We introduce a conditional coarse-to-fine few-shot
learner to refine the coarse predictions made by an efficient object detector,
showing that using an off-the-shelf model leads to poor personalization due to
neural collapse. Therefore, we introduce a Translator block that generates an
auxiliary feature space where features generated by a self-supervised model
(e.g., DINOv2) are distilled without impacting the performance of the detector.
We validate AuXFT on three publicly available datasets and one in-house
benchmark designed for the IPOD task, achieving remarkable gains in all
considered scenarios with excellent time-complexity trade-off: AuXFT reaches a
performance of 80% its upper bound at just 32% of the inference time, 13% of
VRAM and 19% of the model size.

我们设计了一种称为 AuXFT 的少样本 IPOD 策略，通过引入一种条件粗粒度到细粒度的少样本学习器，改善了现有的对象检测器的个性化效果，同时通过 Translator 模块生成辅助特征空间，实现了个性化而不降低检测器的性能。我们在三个公开数据集和一个自行设计的 IPOD 基准测试集上验证了 AuXFT，在所有考虑的情境中都取得了显著的增益，实现了极佳的时间复杂度和性能的权衡：AuXFT 在仅占推理时间的 32％、VRAM 的 13％和模型尺寸的 19％的情况下达到其上限的 80％性能。