Large language models (LLMs) have showcased remarkable prowess in code
generation. However, automated code generation is still challenging since it
requires a high-level semantic mapping between natural language requirements
and codes. Most existing LLMs-based approaches for code generation rely on
decoder-only causal language models often treate codes merely as plain text
tokens \ie feeding the requirements as a prompt input, and outputing code as
flat sequence of tokens, potentially missing the rich semantic features
inherent in source code. To bridge this gap, this paper proposes the "Semantic
Chain-of-Thought" approach to intruduce semantic information of code, named
SeCoT. Our motivation is that the semantic information of the source code (\eg
data flow and control flow) describes more precise program execution behavior,
intention and function. By guiding LLM consider and integrate semantic
information, we can achieve a more granular understanding and representation of
code, enhancing code generation accuracy. Meanwhile, while traditional
techniques leveraging such semantic information require complex static or
dynamic code analysis to obtain features such as data flow and control flow,
SeCoT demonstrates that this process can be fully automated via the intrinsic
capabilities of LLMs (\ie in-context learning), while being generalizable and
applicable to challenging domains. While SeCoT can be applied with different
LLMs, this paper focuses on the powerful GPT-style models: ChatGPT(close-source
model) and WizardCoder(open-source model). The experimental study on three
popular DL benchmarks (\ie HumanEval, HumanEval-ET and MBPP) shows that SeCoT
can achieves state-of-the-art performance, greatly improving the potential for
large models and code generation.

本文提出了一种名为 SeCoT 的语义 “思维链” 方法，通过引入代码的语义信息，指导大型语言模型考虑和整合语义信息，以提高代码生成的准确性。实验结果表明，SeCoT 在大型模型和代码生成方面取得了最先进的性能提升。

编码语义与 LLMs 的桥梁：用于代码生成的语义思维链路提示

Bridging Code Semantic and LLMs: Semantic Chain-of-Thought Prompting for  Code Generation

In natural language processing, transformer-based large language models
(LLMs) like GPT-x models developed by OpenAI have revolutionized the landscape.
Despite their impressive capabilities, these models often encounter challenges
when handling tasks that differ from their training data, resulting in
compromised performance. To address this, few-shot learning has emerged as a
valuable technique, allowing LLMs to adapt with minimal task-specific data. One
innovative strategy, known as Chain-of-Thought Prompting (CoT), has been
introduced to guide LLMs in revealing cognitive processes during multi-step
reasoning. In this paper, we propose Code Chain-of-Thought~(CodeCoT), which
consists of two components: the Vanilla CodeCoT and the Self-exam CodeCoT. The
latter incorporates self-examination, empowering the model to iteratively
generate code, formulate test cases, and refine its outputs. Specifically, the
process entails the generation of test examples by the model corresponding to
the code it is tasked to implement. If it fails on the test examples, then it
regenerates the code based on the erroneous code and associated error types.
Through comprehensive experiments, we observed that both techniques
significantly enhance code generation accuracy across various LLM variants. Our
evaluation results reveal that CodeCoT improves the code generation
effectiveness, including an unprecedented pass@1 accuracy of 79.27\% using the
Self-exam CodeCoT approach on the gpt-3.5-turbo-0613 model in the HumanEval
dataset.

采用自我审查和连续推导的方法，我们提出了一种名为 Code Chain-of-Thought（CodeCoT）的新策略，通过辅助 LLM 模型生成代码和改进输出，从而显著提高了代码生成的准确性。