Learning vectors that capture the meaning of concepts remains a fundamental
challenge. Somewhat surprisingly, perhaps, pre-trained language models have
thus far only enabled modest improvements to the quality of such concept
embeddings. Current strategies for using language models typically represent a
concept by averaging the contextualised representations of its mentions in some
corpus. This is potentially sub-optimal for at least two reasons. First,
contextualised word vectors have an unusual geometry, which hampers downstream
tasks. Second, concept embeddings should capture the semantic properties of
concepts, whereas contextualised word vectors are also affected by other
factors. To address these issues, we propose two contrastive learning
strategies, based on the view that whenever two sentences reveal similar
properties, the corresponding contextualised vectors should also be similar.
One strategy is fully unsupervised, estimating the properties which are
expressed in a sentence from the neighbourhood structure of the contextualised
word embeddings. The second strategy instead relies on a distant supervision
signal from ConceptNet. Our experimental results show that the resulting
vectors substantially outperform existing concept embeddings in predicting the
semantic properties of concepts, with the ConceptNet-based strategy achieving
the best results. These findings are furthermore confirmed in a clustering task
and in the downstream task of ontology completion.

提出通过对比学习策略来改进概念嵌入的语义表示方法，使用全新的 contextualized vectors 代替传统平均表示方法，可优化概念嵌入中的语义属性，使得使用该方法后的概念嵌入能在预测语义属性上显著突出于传统概念嵌入方法，尤其是使用 ConceptNet 的方法的效果最佳。