Large language models (LLMs) have undergone rapid evolution and achieved
remarkable results in recent times. OpenAI's ChatGPT, backed by GPT-3.5 or
GPT-4, has gained instant popularity due to its strong capability across a wide
range of tasks, including natural language tasks, coding, mathematics, and
engaging conversations. However, the impacts and limits of such LLMs in system
security domain are less explored. In this paper, we delve into the limits of
LLMs (i.e., ChatGPT) in seven software security applications including
vulnerability detection/repair, debugging, debloating, decompilation, patching,
root cause analysis, symbolic execution, and fuzzing. Our exploration reveals
that ChatGPT not only excels at generating code, which is the conventional
application of language models, but also demonstrates strong capability in
understanding user-provided commands in natural languages, reasoning about
control and data flows within programs, generating complex data structures, and
even decompiling assembly code. Notably, GPT-4 showcases significant
improvements over GPT-3.5 in most security tasks. Also, certain limitations of
ChatGPT in security-related tasks are identified, such as its constrained
ability to process long code contexts.

研究论文对大型语言模型（LLMs）在系统安全领域中的影响和局限性进行了探讨，发现 ChatGPT 不仅在生成代码方面有出色的能力，还在理解用户提供的自然语言指令、推理程序的控制和数据流、生成复杂数据结构以及反汇编汇编代码等方面展示了强大的能力。研究表明，GPT-4 在大多数安全任务中相较于 GPT-3.5 有显著改进，同时也确定了 ChatGPT 在安全相关任务中的某些限制，例如处理长代码上下文的能力受限。