Pre-trained models (PTMs) have achieved great success in various Software
Engineering (SE) downstream tasks following the ``pre-train then fine-tune''
paradigm. As fully fine-tuning all parameters of PTMs can be computationally
expensive, a widely used solution is parameter-efficient fine-tuning (PEFT),
which freezes PTMs while introducing extra parameters. Though work has been
done to test PEFT methods in the SE field, a comprehensive evaluation is still
lacking. This paper aims to fill in this gap by evaluating the effectiveness of
five PEFT methods on eight PTMs and four SE downstream tasks. For different
tasks and PEFT methods, we seek answers to the following research questions: 1)
Is it more effective to use PTMs trained specifically on source code, or is it
sufficient to use PTMs trained on natural language text? 2) What is the impact
of varying model sizes? 3) How does the model architecture affect the
performance? Besides effectiveness, we also discuss the efficiency of PEFT
methods, concerning the costs of required training time and GPU resource
consumption. We hope that our findings can provide a deeper understanding of
PEFT methods on various PTMs and SE downstream tasks. All the codes and data
are available at https://github.com/zwtnju/PEFT.git.

基于预训练模型的参数高效调整方法在软件工程领域的效果与效率的全面评估研究。