Representation learning for images has been advanced by recent progress in
more complex neural models such as the Vision Transformers and new learning
theories such as the structural causal models. However, these models mainly
rely on the classification loss to implicitly regularize the class-level data
distributions, and they may face difficulties when handling classes with
diverse visual patterns. We argue that the incorporation of the structural
information between data samples may improve this situation. To achieve this
goal, this paper presents a framework termed \textbf{C}lass-level Structural
Relation Modeling and Smoothing for Visual Representation Learning (CSRMS),
which includes the Class-level Relation Modelling, Class-aware Graph Sampling,
and Relational Graph-Guided Representation Learning modules to model a
relational graph of the entire dataset and perform class-aware smoothing and
regularization operations to alleviate the issue of intra-class visual
diversity and inter-class similarity. Specifically, the Class-level Relation
Modelling module uses a clustering algorithm to learn the data distributions in
the feature space and identify three types of class-level sample relations for
the training set; Class-aware Graph Sampling module extends typical training
batch construction process with three strategies to sample dataset-level
sub-graphs; and Relational Graph-Guided Representation Learning module employs
a graph convolution network with knowledge-guided smoothing operations to ease
the projection from different visual patterns to the same class. Experiments
demonstrate the effectiveness of structured knowledge modelling for enhanced
representation learning and show that CSRMS can be incorporated with any
state-of-the-art visual representation learning models for performance gains.
The source codes and demos have been released at
this https URL

本文提出了一个名为 CSRMS 的框架，包括类级关系建模、类感知图采样和关系图引导表示学习模块，以建模数据集的关系图，并执行类感知平滑和正则化操作以减轻类内视觉多样性和类间相似度问题。实验证明结构化知识建模对增强表示学习具有有效性，并表明 CSRMS 可以与任何最先进的视觉表示学习模型结合以提高性能。