We propose PromptBERT, a novel contrastive learning method for learning
better sentence representation. We firstly analyze the drawback of current
sentence embedding from original BERT and find that it is mainly due to the
static token embedding bias and ineffective BERT layers. Then we propose the
first prompt-based sentence embeddings method and discuss two prompt
representing methods and three prompt searching methods to make BERT achieve
better sentence embeddings. Moreover, we propose a novel unsupervised training
objective by the technology of template denoising, which substantially shortens
the performance gap between the supervised and unsupervised settings. Extensive
experiments show the effectiveness of our method. Compared to SimCSE,
PromptBert achieves 2.29 and 2.58 points of improvement based on BERT and
RoBERTa in the unsupervised setting.

本文提出了一种新的对比学习方法 PromptBERT，用于改进句子表示的效果，在分析当前 BERT 句子嵌入的缺陷后，通过两种提示表示方法和三种提示搜索方法实现更好的句子嵌入，同时通过模板去噪技术，提出了一种新的无监督训练目标，大大缩短了有监督和无监督设置之间的性能差距，实验证明了方法的有效性。