Graph-level contrastive learning, aiming to learn the representations for
each graph by contrasting two augmented graphs, has attracted considerable
attention. Previous studies usually simply assume that a graph and its
augmented graph as a positive pair, otherwise as a negative pair. However, it
is well known that graph structure is always complex and multi-scale, which
gives rise to a fundamental question: after graph augmentation, will the
previous assumption still hold in reality? By an experimental analysis, we
discover the semantic information of an augmented graph structure may be not
consistent as original graph structure, and whether two augmented graphs are
positive or negative pairs is highly related with the multi-scale structures.
Based on this finding, we propose a multi-scale subgraph contrastive learning
method which is able to characterize the fine-grained semantic information.
Specifically, we generate global and local views at different scales based on
subgraph sampling, and construct multiple contrastive relationships according
to their semantic associations to provide richer self-supervised signals.
Extensive experiments and parametric analysis on eight graph classification
real-world datasets well demonstrate the effectiveness of the proposed method.

通过对子图取样，在不同尺度上生成全局和局部视图，并根据它们的语义关联构建多种对比关系，以提供更丰富的自监督信号，从而实现对细粒度语义信息的表征。在八个图分类现实世界数据集上进行的大量实验和参数分析充分证明了该方法的有效性。

多尺度子图对比学习

Multi-Scale Subgraph Contrastive Learning

Unsupervised person re-identification (Re-Id) has attracted increasing
attention due to its practical application in the read-world video surveillance
system. The traditional unsupervised Re-Id are mostly based on the method
alternating between clustering and fine-tuning with the classification or
metric learning objectives on the grouped clusters. However, since person Re-Id
is an open-set problem, the clustering based methods often leave out lots of
outlier instances or group the instances into the wrong clusters, thus they can
not make full use of the training samples as a whole. To solve these problems,
we present the hybrid dynamic cluster contrast and probability distillation
algorithm. It formulates the unsupervised Re-Id problem into an unified
local-to-global dynamic contrastive learning and self-supervised probability
distillation framework. Specifically, the proposed method can make the utmost
of the self-supervised signals of all the clustered and un-clustered instances,
from both the instances' self-contrastive level and the probability
distillation respective, in the memory-based non-parametric manner. Besides,
the proposed hybrid local-to-global contrastive learning can take full
advantage of the informative and valuable training examples for effective and
robust training. Extensive experiment results show that the proposed method
achieves superior performances to state-of-the-art methods, under both the
purely unsupervised and unsupervised domain adaptation experiment settings.

本文提出了一种混合动态聚类对比和概率蒸馏算法，在不需要标注数据的情况下，将无监督的人物重识别问题转化为统一的局部到全局动态对比学习和自监督概率蒸馏框架。实验结果表明，这种方法在无监督领域适应性实验设置下，取得了优异的性能。

混合动态对比度和概率蒸馏用于无监督人员再识别

Hybrid Dynamic Contrast and Probability Distillation for Unsupervised  Person Re-Id

Pre-trained text encoders have drawn sustaining attention in natural language
processing (NLP) and shown their capability in obtaining promising results in
different tasks. Recent studies illustrated that external self-supervised
signals (or knowledge extracted by unsupervised learning, such as n-grams) are
beneficial to provide useful semantic evidence for understanding languages such
as Chinese, so as to improve the performance on various downstream tasks
accordingly. To further enhance the encoders, in this paper, we propose to
pre-train n-gram-enhanced encoders with a large volume of data and advanced
techniques for training. Moreover, we try to extend the encoder to different
languages as well as different domains, where it is confirmed that the same
architecture is applicable to these varying circumstances and new
state-of-the-art performance is observed from a long list of NLP tasks across
languages and domains.

本研究提出了使用大规模数据和先进技术针对 n-gram 的预训练编码器，并成功将其拓展到不同语言和领域中，得到了在多项 NLP 任务中表现优异的结果。

ZEN 2.0: N-Gram 增强文本编码器的继续训练和适应

ZEN 2.0: Continue Training and Adaption for N-gram Enhanced Text  Encoders

A common approach to solving physical reasoning tasks is to train a value
learner on example tasks. A limitation of such an approach is that it requires
learning about object dynamics solely from reward values assigned to the final
state of a rollout of the environment. This study aims to address this
limitation by augmenting the reward value with self-supervised signals about
object dynamics. Specifically, we train the model to characterize the
similarity of two environment rollouts, jointly with predicting the outcome of
the reasoning task. This similarity can be defined as a distance measure
between the trajectory of objects in the two rollouts, or learned directly from
pixels using a contrastive formulation. Empirically, we find that this approach
leads to substantial performance improvements on the PHYRE benchmark for
physical reasoning (Bakhtin et al., 2019), establishing a new state-of-the-art.

通过自我监督的信号增强奖励价值，将模型训练出了刻画两个物理场景间相似度的能力，从而在 PHYRE 基准测试中显著提升了表现。