Many recent successes in sentence representation learning have been achieved
by simply fine-tuning on the Natural Language Inference (NLI) datasets with
triplet loss or siamese loss. Nevertheless, they share a common weakness:
sentences in a contradiction pair are not necessarily from different semantic
categories. Therefore, optimizing the semantic entailment and contradiction
reasoning objective alone is inadequate to capture the high-level semantic
structure. The drawback is compounded by the fact that the vanilla siamese or
triplet losses only learn from individual sentence pairs or triplets, which
often suffer from bad local optima. In this paper, we propose PairSupCon, an
instance discrimination based approach aiming to bridge semantic entailment and
contradiction understanding with high-level categorical concept encoding. We
evaluate PairSupCon on various downstream tasks that involve understanding
sentence semantics at different granularities. We outperform the previous
state-of-the-art method with $10\%$--$13\%$ averaged improvement on eight
clustering tasks, and $5\%$--$6\%$ averaged improvement on seven semantic
textual similarity (STS) tasks.

本文提出了一种基于实例区分的方法 PairSupCon，旨在通过高级分类概念编码在语义蕴含和矛盾理解之间建立联系。该方法在各种下游任务中均取得了极好的效果，超过了先前的最先进方法。