Researchers have explored the potential of utilizing pre-trained language
models, such as CodeBERT, to improve source code-related tasks. Previous
studies have mainly relied on CodeBERT's text embedding capability and the
`[CLS]' sentence embedding information as semantic representations for
fine-tuning downstream source code-related tasks. However, these methods
require additional neural network layers to extract effective features,
resulting in higher computational costs. Furthermore, existing approaches have
not leveraged the rich knowledge contained in both source code and related
text, which can lead to lower accuracy. This paper presents a novel approach,
CodePrompt, which utilizes rich knowledge recalled from a pre-trained model by
prompt learning and an attention mechanism to improve source code-related
classification tasks. Our approach initially motivates the language model with
prompt information to retrieve abundant knowledge associated with the input as
representative features, thus avoiding the need for additional neural network
layers and reducing computational costs. Subsequently, we employ an attention
mechanism to aggregate multiple layers of related knowledge for each task as
final features to boost their accuracy. We conducted extensive experiments on
four downstream source code-related tasks to evaluate our approach and our
results demonstrate that CodePrompt achieves new state-of-the-art performance
on the accuracy metric while also exhibiting computation cost-saving
capabilities.

CodePrompt 是一种新颖方法，利用 prompt 学习和注意机制从预训练模型中召回丰富的知识来提高源代码相关的分类任务的准确性。该方法避免了需要额外的神经网络层和降低计算成本，并采用注意机制来聚合多层相关知识作为最终特征，从而在准确度方面达到新的最佳性能，并具备节省计算成本的能力。