What is an image and how to extract latent features? Convolutional Networks
(ConvNets) consider an image as organized pixels in a rectangular shape and
extract features via convolutional operation in local region; Vision
Transformers (ViTs) treat an image as a sequence of patches and extract
features via attention mechanism in a global range. In this work, we introduce
a straightforward and promising paradigm for visual representation, which is
called Context Clusters. Context clusters (CoCs) view an image as a set of
unorganized points and extract features via simplified clustering algorithm. In
detail, each point includes the raw feature (e.g., color) and positional
information (e.g., coordinates), and a simplified clustering algorithm is
employed to group and extract deep features hierarchically. Our CoCs are
convolution- and attention-free, and only rely on clustering algorithm for
spatial interaction. Owing to the simple design, we show CoCs endow gratifying
interpretability via the visualization of clustering process. Our CoCs aim at
providing a new perspective on image and visual representation, which may enjoy
broad applications in different domains and exhibit profound insights. Even
though we are not targeting SOTA performance, COCs still achieve comparable or
even better results than ConvNets or ViTs on several benchmarks. Codes are
available at: this https URL

本文介绍了一种新的视觉表示范式 - 环境群聚 (Context Clusters)，可通过简化的聚类算法在一组未组织的点中提取特征，无需卷积或注意力机制，并提供了可视化聚类过程的令人满意的可解释性。在几个基准测试中，Context Clusters 可达到与 ConvNets 或 Vision Transformers 相当甚至更好的结果。