Grammatical error correction aims to correct ungrammatical sentences
automatically. Recently, some work has demonstrated the excellent capabilities
of closed-source Large Language Models (LLMs, e.g., ChatGPT) in grammatical
error correction. However, the potential of open-source LLMs remains
unexplored. In this paper, we introduced GrammarGPT, an open-source LLM, to
preliminary explore its potential for native Chinese grammatical error
correction. The core recipe of GrammarGPT is to leverage the hybrid dataset of
ChatGPT-generated and human-annotated. For grammatical errors with clues, we
proposed a heuristic method to guide ChatGPT to generate ungrammatical
sentences by providing those clues. For grammatical errors without clues, we
collected ungrammatical sentences from publicly available websites and manually
corrected them. In addition, we employed an error-invariant augmentation method
to enhance the ability of the model to correct native Chinese grammatical
errors. We ultimately constructed about 1k parallel data and utilized these
data to fine-tune open-source LLMs (e.g., Phoenix, released by The Chinese
University of Hong Kong, Shenzhen) with instruction tuning. The experimental
results show that GrammarGPT outperforms the existing SOTA system
significantly. Although model parameters are 20x larger than the SOTA baseline,
the required amount of data for instruction tuning is 1200x smaller,
illustrating the potential of open-source LLMs on native CGEC. Our GrammarGPT
ranks $3^{rd}$ on NLPCC2023 SharedTask1, demonstrating our approach's
effectiveness. The code and data are available at
https://github.com/FreedomIntelligence/GrammarGPT.

介绍了 GrammarGPT，一款开源的大型语言模型，通过使用混合数据集和引导方法，提高了中文语法纠错的性能，最终在 NLPCC2023 SharedTask1 中取得了第三名的成绩。