Class-agnostic object counting aims to count all objects in an image with
respect to example boxes or class names, \emph{a.k.a} few-shot and zero-shot
counting. Current state-of-the-art methods highly rely on density maps to
predict object counts, which lacks model interpretability. In this paper, we
propose a generalized framework for both few-shot and zero-shot object counting
based on detection. Our framework combines the superior advantages of two
foundation models without compromising their zero-shot capability: (\textbf{i})
SAM to segment all possible objects as mask proposals, and (\textbf{ii}) CLIP
to classify proposals to obtain accurate object counts. However, this strategy
meets the obstacles of efficiency overhead and the small crowded objects that
cannot be localized and distinguished. To address these issues, our framework,
termed PseCo, follows three steps: point, segment, and count. Specifically, we
first propose a class-agnostic object localization to provide accurate but
least point prompts for SAM, which consequently not only reduces computation
costs but also avoids missing small objects. Furthermore, we propose a
generalized object classification that leverages CLIP image/text embeddings as
the classifier, following a hierarchical knowledge distillation to obtain
discriminative classifications among hierarchical mask proposals. Extensive
experimental results on FSC-147 dataset demonstrate that PseCo achieves
state-of-the-art performance in both few-shot/zero-shot object
counting/detection, with additional results on large-scale COCO and LVIS
datasets. The source code is available at
https://github.com/Hzzone/PseCo.

基于检测的 PseCo 框架将 SAM 和 CLIP 结合起来，实现了对少样本和零样本的准确目标计数，通过点定位、分割和计数等步骤解决了效率和小物体区分等难题。

点、线段和计数：一种用于对象计数的普遍框架

Point, Segment and Count: A Generalized Framework for Object Counting

Class-agnostic object counting aims to count object instances of an arbitrary
class at test time. It is challenging but also enables many potential
applications. Current methods require human-annotated exemplars as inputs which
are often unavailable for novel categories, especially for autonomous systems.
Thus, we propose zero-shot object counting (ZSC), a new setting where only the
class name is available during test time. This obviates the need for human
annotators and enables automated operation. To perform ZSC, we propose finding
a few object crops from the input image and use them as counting exemplars. The
goal is to identify patches containing the objects of interest while also being
visually representative for all instances in the image. To do this, we first
construct class prototypes using large language-vision models, including CLIP
and Stable Diffusion, to select the patches containing the target objects.
Furthermore, we propose a ranking model that estimates the counting error of
each patch to select the most suitable exemplars for counting. Experimental
results on a recent class-agnostic counting dataset, FSC-147, validate the
effectiveness of our method.

零样本目标计数 (ZSC) 是一种新的设置，只需类名作为测试时间的输入。我们提出了找到几个目标裁剪样本并将其用作计数示例的方法，该方法消除了对人工注释者的需求并实现了自动化操作。通过使用大型语言 - 视觉模型构建类原型，我们选择包含目标对象的补丁，同时使用排名模型评估每个补丁的计数误差并选择最合适的示例用于计数。在最近的类别无关计数数据集 FSC-147 上的实验结果验证了我们方法的有效性。

语言 - 视觉模型下的零样本物体计数

Zero-Shot Object Counting with Language-Vision Models

Class-agnostic object counting aims to count object instances of an arbitrary
class at test time. It is challenging but also enables many potential
applications. Current methods require human-annotated exemplars as inputs which
are often unavailable for novel categories, especially for autonomous systems.
Thus, we propose zero-shot object counting (ZSC), a new setting where only the
class name is available during test time. Such a counting system does not
require human annotators in the loop and can operate automatically. Starting
from a class name, we propose a method that can accurately identify the optimal
patches which can then be used as counting exemplars. Specifically, we first
construct a class prototype to select the patches that are likely to contain
the objects of interest, namely class-relevant patches. Furthermore, we
introduce a model that can quantitatively measure how suitable an arbitrary
patch is as a counting exemplar. By applying this model to all the candidate
patches, we can select the most suitable patches as exemplars for counting.
Experimental results on a recent class-agnostic counting dataset, FSC-147,
validate the effectiveness of our method. Code is available at
this https URL

本文提出了零样本目标计数（ZSC）这样一种新的设置，该计数系统不需要在循环中进行人工注释，并且可以自动运行。该方法首先构造一个类原型来选择可能包含感兴趣对象即类相关块的补丁，并引入一种可以定量衡量任意补丁作为计数示例的适宜程度的模型，通过将该模型应用于所有候选补丁，可以选择最合适的补丁作为计数示例，实验结果验证了方法的有效性。