Supervised contrastive learning (SCL) frameworks treat each class as
independent and thus consider all classes to be equally important. This
neglects the common scenario in which label hierarchy exists, where
fine-grained classes under the same category show more similarity than very
different ones. This paper introduces a family of Label-Aware SCL methods
(LASCL) that incorporates hierarchical information to SCL by leveraging
similarities between classes, resulting in creating a more well-structured and
discriminative feature space. This is achieved by first adjusting the distance
between instances based on measures of the proximity of their classes with the
scaled instance-instance-wise contrastive. An additional instance-center-wise
contrastive is introduced to move within-class examples closer to their
centers, which are represented by a set of learnable label parameters. The
learned label parameters can be directly used as a nearest neighbor classifier
without further finetuning. In this way, a better feature representation is
generated with improvements of intra-cluster compactness and inter-cluster
separation. Experiments on three datasets show that the proposed LASCL works
well on text classification of distinguishing a single label among
multi-labels, outperforming the baseline supervised approaches. Our code is
publicly available.

我们介绍了一种 Label-Aware SCL 方法，通过利用类别之间的相似性，将层次信息融入到 SCL 中，从而创建一个更结构化和具有辨别性的特征空间，实现了更好的文本分类性能。