While GPTs with traditional fine-tuning fail to achieve strong results on
natural language understanding (NLU), we show that GPTs can be better than or
comparable to similar-sized BERTs on NLU tasks with a novel method P-tuning --
which employs trainable continuous prompt embeddings. On the knowledge probing
(LAMA) benchmark, the best GPT recovers 64\% (P@1) of world knowledge without
any additional text provided during test time, which substantially improves the
previous best by 20+ percentage points. On the SuperGlue benchmark, GPTs
achieve comparable and sometimes better performance to similar-sized BERTs in
supervised learning. Importantly, we find that P-tuning also improves BERTs'
performance in both few-shot and supervised settings while largely reducing the
need for prompt engineering. Consequently, P-tuning outperforms the
state-of-the-art approaches on the few-shot SuperGlue benchmark.

利用可训练的连续提示嵌入 (P-tuning) 方法，在自然语言理解 (NLU) 任务上，GPT 在不需要额外文本的情况下，能够比同等大小的 BERT 表现更好或者相当，并且在 LAMA 基准测试中恢复了 64％的世界知识，同时在监督学习中，GPT 表现出类似大小的 BERT 相似或更好的性能。此外，该方法还显著减少了对提示工程的需求，并在少样本 SuperGlue 基准测试中优于现有最先进方法。