Hierarchical semantic structures naturally exist in an image dataset, in
which several semantically relevant image clusters can be further integrated
into a larger cluster with coarser-grained semantics. Capturing such structures
with image representations can greatly benefit the semantic understanding on
various downstream tasks. Existing contrastive representation learning methods
lack such an important model capability. In addition, the negative pairs used
in these methods are not guaranteed to be semantically distinct, which could
further hamper the structural correctness of learned image representations. To
tackle these limitations, we propose a novel contrastive learning framework
called Hierarchical Contrastive Selective Coding (HCSC). In this framework, a
set of hierarchical prototypes are constructed and also dynamically updated to
represent the hierarchical semantic structures underlying the data in the
latent space. To make image representations better fit such semantic
structures, we employ and further improve conventional instance-wise and
prototypical contrastive learning via an elaborate pair selection scheme. This
scheme seeks to select more diverse positive pairs with similar semantics and
more precise negative pairs with truly distinct semantics. On extensive
downstream tasks, we verify the superior performance of HCSC over
state-of-the-art contrastive methods, and the effectiveness of major model
components is proved by plentiful analytical studies. We build a comprehensive
model zoo in Sec. D. Our source code and model weights are available at
this https URL

提出了一种新的对比学习框架 HCSC，可以通过构建并动态更新一组层次结构原型来表征图片数据中的层次语义结构，并且通过一种精心设计的对对选择方案来让图像表示更好地适应这些语义结构。在广泛的下游任务中，证明了 HCSC 优于现有对比方法，其重要模型组件的有效性通过大量的分析研究得到证明。

HCSC: 分层对比选择编码

HCSC: Hierarchical Contrastive Selective Coding

This paper studies unsupervised/self-supervised whole-graph representation
learning, which is critical in many tasks such as molecule properties
prediction in drug and material discovery. Existing methods mainly focus on
preserving the local similarity structure between different graph instances but
fail to discover the global semantic structure of the entire data set. In this
paper, we propose a unified framework called Local-instance and Global-semantic
Learning (GraphLoG) for self-supervised whole-graph representation learning.
Specifically, besides preserving the local similarities, GraphLoG introduces
the hierarchical prototypes to capture the global semantic clusters. An
efficient online expectation-maximization (EM) algorithm is further developed
for learning the model. We evaluate GraphLoG by pre-training it on massive
unlabeled graphs followed by fine-tuning on downstream tasks. Extensive
experiments on both chemical and biological benchmark data sets demonstrate the
effectiveness of the proposed approach.

该论文提出了一种统一框架，名为 GraphLoG，用于自监督整图表示学习，其除保留本地相似性外，还引入了分层原型来捕捉全局语义簇，进一步发展了一个高效的在线期望最大化算法来学习该模型，并在化学和生物基准数据集上进行了广泛实验，证明所提出的方法的有效性。

使用本地和全局结构进行自监督的图级表示学习

Self-supervised Graph-level Representation Learning with Local and  Global Structure

Vision models are interpretable when they classify objects on the basis of
features that a person can directly understand. Recently, methods relying on
visual feature prototypes have been developed for this purpose. However, in
contrast to how humans categorize objects, these approaches have not yet made
use of any taxonomical organization of class labels. With such an approach, for
instance, we may see why a chimpanzee is classified as a chimpanzee, but not
why it was considered to be a primate or even an animal. In this work we
introduce a model that uses hierarchically organized prototypes to classify
objects at every level in a predefined taxonomy. Hence, we may find distinct
explanations for the prediction an image receives at each level of the
taxonomy. The hierarchical prototypes enable the model to perform another
important task: interpretably classifying images from previously unseen classes
at the level of the taxonomy to which they correctly relate, e.g. classifying a
hand gun as a weapon, when the only weapons in the training data are rifles.
With a subset of ImageNet, we test our model against its counterpart black-box
model on two tasks: 1) classification of data from familiar classes, and 2)
classification of data from previously unseen classes at the appropriate level
in the taxonomy. We find that our model performs approximately as well as its
counterpart black-box model while allowing for each classification to be
interpreted.

通过使用根据预定义的分类法层次化组织的视觉原型模型，可以实现解释性地对图像进行分类，使得该模型能够对来自先前未见过的类别的图像进行解释性地分类。