The next era of program understanding is being propelled by the use of
machine learning to solve software problems. Recent studies have shown
surprising results of source code learning, which applies deep neural networks
(DNNs) to various critical software tasks, e.g., bug detection and clone
detection. This success can be greatly attributed to the utilization of massive
high-quality training data, and in practice, data augmentation, which is a
technique used to produce additional training data, has been widely adopted in
various domains, such as computer vision. However, in source code learning,
data augmentation has not been extensively studied, and existing practice is
limited to simple syntax-preserved methods, such as code refactoring.
Essentially, source code is often represented in two ways, namely, sequentially
as text data and structurally as graph data, when it is used as training data
in source code learning. Inspired by these analogy relations, we take an early
step to investigate whether data augmentation methods that are originally used
for text and graphs are effective in improving the training quality of source
code learning. To that end, we first collect and categorize data augmentation
methods in the literature. Second, we conduct a comprehensive empirical study
on four critical tasks and 11 DNN architectures to explore the effectiveness of
12 data augmentation methods (including code refactoring and 11 other methods
for text and graph data). Our results identify the data augmentation methods
that can produce more accurate and robust models for source code learning,
including those based on mixup (e.g., SenMixup for texts and Manifold-Mixup for
graphs), and those that slightly break the syntax of source code (e.g., random
swap and random deletion for texts).

本篇研究探讨使用数据增强方法，如 mixup 和 syntax-breaking，对于用于源代码语言的训练数据的质量提升，结果表明这些方法可以产生更准确和稳健的模型。

数据增强在源代码学习中的应用：一项实证研究

Boosting Source Code Learning with Data Augmentation: An Empirical Study

Distribution shift has been a longstanding challenge for the reliable
deployment of deep learning (DL) models due to unexpected accuracy degradation.
Although DL has been becoming a driving force for large-scale source code
analysis in the big code era, limited progress has been made on distribution
shift analysis and benchmarking for source code tasks. To fill this gap, this
paper initiates to propose CodeS, a distribution shift benchmark dataset, for
source code learning. Specifically, CodeS supports two programming languages
(Java and Python) and five shift types (task, programmer, time-stamp, token,
and concrete syntax tree). Extensive experiments based on CodeS reveal that 1)
out-of-distribution detectors from other domains (e.g., computer vision) do not
generalize to source code, 2) all code classification models suffer from
distribution shifts, 3) representation-based shifts have a higher impact on the
model than others, and 4) pre-trained bimodal models are relatively more
resistant to distribution shifts.

本文提出了 CodeS, 一个用于源代码学习的分布式模型的测试基准数据集，实验证明，在源代码任务中，其他领域的模型无法普遍适用，所有的代码分类模型都存在分布转变且受到表示性分布的影响更大，而预训练的双模型相对分布分离的适应性较强。