Greybox fuzzing has achieved success in revealing bugs and vulnerabilities in
programs. However, randomized mutation strategies have limited the fuzzer's
performance on structured data. Specialized fuzzers can handle complex
structured data, but require additional efforts in grammar and suffer from low
throughput.
In this paper, we explore the potential of utilizing the Large Language Model
to enhance greybox fuzzing for structured data. We utilize the pre-trained
knowledge of LLM about data conversion and format to generate new valid inputs.
We further fine-tuned it with paired mutation seeds to learn structured format
and mutation strategies effectively. Our LLM-based fuzzer, LLAMAFUZZ,
integrates the power of LLM to understand and mutate structured data to
fuzzing. We conduct experiments on the standard bug-based benchmark Magma and a
wide variety of real-world programs. LLAMAFUZZ outperforms our top competitor
by 41 bugs on average. We also identified 47 unique bugs across all trials.
Moreover, LLAMAFUZZ demonstrated consistent performance on both bug trigger and
bug reached. Compared to AFL++, LLAMAFUZZ achieved 27.19% more branches in
real-world program sets on average. We also demonstrate a case study to explain
how LLMs enhance the fuzzing process in terms of code coverage.

利用预训练的大型语言模型 (LLM) 增强结构化数据的灰盒模糊测试，通过利用 LLM 的先前训练知识和基于 Magma 等基准的实验，LLAMAFUZZ 比其他方法平均提高了 41 个漏洞的检测，实现了一致的性能提升。