Pretrained Transformers achieve state-of-the-art performance in various
code-processing tasks but may be too large to be deployed. As software
development tools often incorporate modules for various purposes which may
potentially use a single instance of the pretrained model, it appears relevant
to utilize parameter-efficient fine-tuning for the pretrained models of code.
In this work, we test two widely used approaches, adapters and LoRA, which were
initially tested on NLP tasks, on four code-processing tasks. We find that
though the efficient fine-tuning approaches may achieve comparable or higher
performance than the standard, full, fine-tuning in code understanding tasks,
they underperform full fine-tuning in code-generative tasks. These results
underline the importance of testing efficient fine-tuning approaches on other
domains than NLP and motivate future research in efficient fine-tuning for
source code.

本文介绍了针对代码的预训练模型参数高效微调的适配器和 LoRA 两种方法，经过四项代码处理任务测试，发现这些高效微调方法在代码理解任务中可以实现可比或更高的性能，但在代码生成任务中性能不如标准的全微调。这些结果强调了在其他领域测试高效微调方法的重要性，并促进了对源代码高效微调的未来研究。