Few-shot detection is a major task in pattern recognition which seeks to
localize objects using models trained with few labeled data. One of the
mainstream few-shot methods is transfer learning which consists in pretraining
a detection model in a source domain prior to its fine-tuning in a target
domain. However, it is challenging for fine-tuned models to effectively
identify new classes in the target domain, particularly when the underlying
labeled training data are scarce. In this paper, we devise a novel sparse
context transformer (SCT) that effectively leverages object knowledge in the
source domain, and automatically learns a sparse context from only few training
images in the target domain. As a result, it combines different relevant clues
in order to enhance the discrimination power of the learned detectors and
reduce class confusion. We evaluate the proposed method on two challenging
few-shot object detection benchmarks, and empirical results show that the
proposed method obtains competitive performance compared to the related
state-of-the-art.

本文介绍了一种利用稀疏上下文变换器（SCT）的新方法，在目标域仅有少量训练图像的情况下，有效地利用源域的物体知识，并自动从目标域中学习稀疏上下文，通过结合不同相关线索来增强学习检测器的区分能力并降低类别混淆。两个具有挑战性的少样本目标检测基准测试结果表明，该方法与相关最新技术相比具有竞争性能。

稀疏上下文变换的小样本目标检测

Few-Shot Object Detection with Sparse Context Transformers

Large language models (LLMs)-based image captioning has the capability of
describing objects not explicitly observed in training data; yet novel objects
occur frequently, necessitating the requirement of sustaining up-to-date object
knowledge for open-world comprehension. Instead of relying on large amounts of
data and scaling up network parameters, we introduce a highly effective
retrieval-augmented image captioning method that prompts LLMs with object names
retrieved from External Visual--name memory (EVCap). We build ever-changing
object knowledge memory using objects' visuals and names, enabling us to (i)
update the memory at a minimal cost and (ii) effortlessly augment LLMs with
retrieved object names utilizing a lightweight and fast-to-train model. Our
model, which was trained only on the COCO dataset, can be adapted to out-domain
data without additional fine-tuning or retraining. Our comprehensive
experiments conducted on various benchmarks and synthetic commonsense-violating
data demonstrate that EVCap, comprising solely 3.97M trainable parameters,
exhibits superior performance compared to other methods of equivalent model
size scale. Notably, it achieves competitive performance against specialist
SOTAs with an enormous number of parameters. Our code is available at
this https URL

该研究介绍了一种基于大型语言模型的图像字幕生成方法，利用外部可视 - 名称记忆检索并提供对象名称，从而更新对象知识记忆，实现对未知对象的理解，并在各种基准测试中展现出优越性能。

EVCap：用于开放世界理解的外部视觉名称存储的检索增强图像字幕

EVCap: Retrieval-Augmented Image Captioning with External Visual-Name  Memory for Open-World Comprehension

Existing language and vision models achieve impressive performance in
image-text understanding. Yet, it is an open question to what extent they can
be used for language understanding in 3D environments and whether they
implicitly acquire 3D object knowledge, e.g. about different views of an
object. In this paper, we investigate whether a state-of-the-art language and
vision model, CLIP, is able to ground perspective descriptions of a 3D object
and identify canonical views of common objects based on text queries. We
present an evaluation framework that uses a circling camera around a 3D object
to generate images from different viewpoints and evaluate them in terms of
their similarity to natural language descriptions. We find that a pre-trained
CLIP model performs poorly on most canonical views and that fine-tuning using
hard negative sampling and random contrasting yields good results even under
conditions with little available training data.

本论文研究了 CLIP 模型在 3D 环境下对物体视角描述和识别中的表现以及对少量可用训练数据条件下的硬负采样和随机对比进行微调。