Code pre-trained models (CodePTMs) have recently demonstrated a solid
capacity to process various software intelligence tasks, e.g., code clone
detection, code translation, and code summarization. The current mainstream
method that deploys these models to downstream tasks is to fine-tune them on
individual tasks, which is generally costly and needs sufficient data for large
models. To tackle the issue, in this paper, we present TransCoder, a unified
Transferable fine-tuning strategy for Code representation learning. Inspired by
human inherent skills of knowledge generalization, TransCoder drives the model
to learn better code-related meta-knowledge like human programmers.
Specifically, we employ a tunable prefix encoder as the meta-learner to capture
cross-task and cross-language transferable knowledge, respectively. Besides,
tasks with minor training sample sizes and languages with small corpus can be
remarkably benefited from our approach. Extensive experiments conducted on
benchmark datasets clearly demonstrate that our method can lead to superior
performance on various code-related tasks and encourage mutual reinforcement.
We also show that TransCoder is applicable in low-resource scenarios.

本文介绍了 TransCoder，一种统一的用于代码表示学习的可转移微调策略，能够促进代码相关任务的卓越性能，并鼓励相互强化。