With the advancement of deep learning (DL) in various fields, there are many
attempts to reveal software vulnerabilities by data-driven approach.
Nonetheless, such existing works lack the effective representation that can
retain the non-sequential semantic characteristics and contextual relationship
of source code attributes. Hence, in this work, we propose XGV-BERT, a
framework that combines the pre-trained CodeBERT model and Graph Neural Network
(GCN) to detect software vulnerabilities. By jointly training the CodeBERT and
GCN modules within XGV-BERT, the proposed model leverages the advantages of
large-scale pre-training, harnessing vast raw data, and transfer learning by
learning representations for training data through graph convolution. The
research results demonstrate that the XGV-BERT method significantly improves
vulnerability detection accuracy compared to two existing methods such as
VulDeePecker and SySeVR. For the VulDeePecker dataset, XGV-BERT achieves an
impressive F1-score of 97.5%, significantly outperforming VulDeePecker, which
achieved an F1-score of 78.3%. Again, with the SySeVR dataset, XGV-BERT
achieves an F1-score of 95.5%, surpassing the results of SySeVR with an
F1-score of 83.5%.

我们提出了一种名为 XGV-BERT 的框架，结合预训练的 CodeBERT 模型和图神经网络（GCN）来检测软件漏洞，通过联合训练 CodeBERT 和 GCN 模块，该模型利用大规模预训练、通过图卷积学习训练数据的表示，其研究结果表明与 VulDeePecker 和 SySeVR 等现有方法相比，XGV-BERT 方法显著提高了漏洞检测的准确性。