Code generation models are not robust to small perturbations, which often
lead to inconsistent and incorrect generations and significantly degrade the
performance of these models. Improving the robustness of code generation models
is crucial to better user experience when these models are deployed in
real-world applications. However, existing efforts have not addressed this
issue for code generation models. To fill this gap, we propose CodeFort, a
framework to improve the robustness of code generation models, generalizing a
large variety of code perturbations to enrich the training data and enabling
various robust training strategies, mixing data augmentation, batch
augmentation, adversarial logits pairing, and contrastive learning, all
carefully designed to support high-throughput training. Extensive evaluations
show that we improve the average robust pass rates of baseline CodeGen models
from 14.79 to 21.74. Notably, the improvement in robustness against code-syntax
perturbations is evidenced by a significant decrease in pass rate drop from
95.04% to 53.35%

用 CodeFort 框架提高代码生成模型的鲁棒性，通过丰富训练数据并采用多种训练策略，包括混合数据增强、批量数据增强、对抗性 Logits 对齐和对比学习，将基准 CodeGen 模型的平均鲁棒通过率从 14.79% 提高到 21.74%，尤其在对代码语法扰动的鲁棒性上，通过率的降低显著减少了从 95.04% 到 53.35%。