Programming language understanding and representation (a.k.a code
representation learning) has always been a hot and challenging task in software
engineering. It aims to apply deep learning techniques to produce numerical
representations of the source code features while preserving its semantics.
These representations can be used for facilitating subsequent code-related
tasks. The abstract syntax tree (AST), a fundamental code feature, illustrates
the syntactic information of the source code and has been widely used in code
representation learning. However, there is still a lack of systematic and
quantitative evaluation of how well AST-based code representation facilitates
subsequent code-related tasks. In this paper, we first conduct a comprehensive
empirical study to explore the effectiveness of the AST-based code
representation in facilitating follow-up code-related tasks. To do so, we
compare the performance of models trained with code token sequence (Token for
short) based code representation and AST-based code representation on three
popular types of code-related tasks. Surprisingly, the overall quantitative
statistical results demonstrate that models trained with AST-based code
representation consistently perform worse across all three tasks compared to
models trained with Token-based code representation. Our further quantitative
analysis reveals that models trained with AST-based code representation
outperform models trained with Token-based code representation in certain
subsets of samples across all three tasks. We also conduct comprehensive
experiments to evaluate and reveal the impact of the choice of AST
parsing/preprocessing/encoding methods on AST-based code representation and
subsequent code-related tasks. Our study provides future researchers with
detailed guidance on how to select solutions at each stage to fully exploit
AST.

在软件工程领域，编程语言理解与表示（即代码表示学习）一直是一个备受关注和具有挑战性的任务，其旨在应用深度学习技术生成源代码特征的数值表示同时保持其语义，这些表示可用于促进后续与代码相关的任务。然而，AST（抽象语法树）作为一种基本的代码特征，在代码表示学习中广泛应用，但目前缺乏对 AST-based 代码表示如何促进后续代码相关任务的系统和定量评估。本文通过详细的实证研究首次探索了 AST-based 代码表示在促进后续代码相关任务方面的有效性，并比较了基于代码标记序列和基于 AST 的代码表示方法在三种常见类型的代码相关任务上的性能。令人惊讶的是，整体定量统计结果表明，与基于代码标记的代码表示方案相比，基于 AST 的代码表示方法在所有三个任务中的表现均较差。进一步的定量分析显示，基于 AST 的代码表示方法在所有三个任务的某些子样本中优于基于代码标记的代码表示方法。我们还进行了全面的实验，评估和揭示了 AST 解析 / 预处理 / 编码方法的选择对基于 AST 的代码表示和后续代码相关任务的影响。本研究为未来的研究人员提供了关于如何选择每个阶段的解决方案以充分利用 AST 的详细指导。