We investigate whether region-based representations are effective for
recognition. Regions were once a mainstay in recognition approaches, but pixel
and patch-based features are now used almost exclusively. We show that recent
class-agnostic segmenters like SAM can be effectively combined with strong
unsupervised representations like DINOv2 and used for a wide variety of tasks,
including semantic segmentation, object-based image retrieval, and multi-image
analysis. Once the masks and features are extracted, these representations,
even with linear decoders, enable competitive performance, making them well
suited to applications that require custom queries. The compactness of the
representation also makes it well-suited to video analysis and other problems
requiring inference across many images.

我们研究了基于区域的表示方法在识别中的有效性，并展示了最近的类别不可知分割器，如 SAM 结合强无监督表示方法 (DINOv2) 的有效性，以及在语义分割、基于对象的图像检索和多图像分析等各种任务上的应用。提取出的掩模和特征使得即使使用线性解码器，这些表示方法也能实现竞争性能，使其非常适用于需要自定义查询的应用。该表示方法的紧凑性也使其非常适用于视频分析和其他需要在多个图像之间进行推理的问题。

基于区域的再思考

Region-Based Representations Revisited

While many methods for learning vector space embeddings have been proposed in
the field of Natural Language Processing, these methods typically do not
distinguish between categories and individuals. Intuitively, if individuals are
represented as vectors, we can think of categories as (soft) regions in the
embedding space. Unfortunately, meaningful regions can be difficult to
estimate, especially since we often have few examples of individuals that
belong to a given category. To address this issue, we rely on the fact that
different categories are often highly interdependent. In particular, categories
often have conceptual neighbors, which are disjoint from but closely related to
the given category (e.g.\ fruit and vegetable). Our hypothesis is that more
accurate category representations can be learned by relying on the assumption
that the regions representing such conceptual neighbors should be adjacent in
the embedding space. We propose a simple method for identifying conceptual
neighbors and then show that incorporating these conceptual neighbors indeed
leads to more accurate region based representations.

本文介绍了一种将概念邻居纳入考虑的方法来学习区域向量表示，从而得到更加准确的基于区域的表示。