We investigate whether region-based representations are effective for
recognition. Regions were once a mainstay in recognition approaches, but pixel
and patch-based features are now used almost exclusively. We show that recent
class-agnostic segmenters like SAM can be effectively combined with strong
unsupervised representations like DINOv2 and used for a wide variety of tasks,
including semantic segmentation, object-based image retrieval, and multi-image
analysis. Once the masks and features are extracted, these representations,
even with linear decoders, enable competitive performance, making them well
suited to applications that require custom queries. The compactness of the
representation also makes it well-suited to video analysis and other problems
requiring inference across many images.

我们研究了基于区域的表示方法在识别中的有效性，并展示了最近的类别不可知分割器，如 SAM 结合强无监督表示方法 (DINOv2) 的有效性，以及在语义分割、基于对象的图像检索和多图像分析等各种任务上的应用。提取出的掩模和特征使得即使使用线性解码器，这些表示方法也能实现竞争性能，使其非常适用于需要自定义查询的应用。该表示方法的紧凑性也使其非常适用于视频分析和其他需要在多个图像之间进行推理的问题。