Real-world programs expecting structured inputs often has a format-parsing
stage gating the deeper program space. Neither a mutation-based approach nor a
generative approach can provide a solution that is effective and scalable.
Large language models (LLM) pre-trained with an enormous amount of natural
language corpus have proved to be effective for understanding the implicit
format syntax and generating format-conforming inputs. In this paper, propose
ChatFuzz, a greybox fuzzer augmented by generative AI. More specifically, we
pick a seed in the fuzzer's seed pool and prompt ChatGPT generative models to
variations, which are more likely to be format-conforming and thus of high
quality. We conduct extensive experiments to explore the best practice for
harvesting the power of generative LLM models. The experiment results show that
our approach improves the edge coverage by 12.77\% over the SOTA greybox fuzzer
(AFL++) on 12 target programs from three well-tested benchmarks. As for
vulnerability detection, \sys is able to perform similar to or better than
AFL++ for programs with explicit syntax rules but not for programs with
non-trivial syntax.

本文介绍了一种名为 ChatFuzz 的灰盒模糊测试工具，它利用生成式人工智能生成特定格式的输入，以提高输入测试质量和覆盖范围，实验结果表明在特定情况下，该工具能比当前最先进的 AFL++ 模糊测试工具更好地检测程序漏洞。