In recent years, large language models have attracted significant attention
due to their exceptional performance across a multitude of natural language
process tasks, and have been widely applied in various fields. However, the
application of large language models in the Intellectual Property (IP) space is
challenging due to the strong need for specialized knowledge, privacy
protection, processing of extremely long text in this field. In this technical
report, we present for the first time a low-cost, standardized procedure for
training IP-oriented LLMs, meeting the unique requirements of the IP domain.
Using this standard process, we have trained the PatentGPT series models based
on open-source pretrained models. By evaluating them on the open-source
IP-oriented benchmark MOZIP, our domain-specific LLMs outperforms GPT-4,
indicating the effectiveness of the proposed training procedure and the
expertise of the PatentGPT models in the IP demain. What is impressive is that
our model significantly outperformed GPT-4 on the 2019 China Patent Agent
Qualification Examination by achieving a score of 65, reaching the level of
human experts. Additionally, the PatentGPT model, which utilizes the SMoE
architecture, achieves performance comparable to that of GPT-4 in the IP domain
and demonstrates a better cost-performance ratio on long-text tasks,
potentially serving as an alternative to GPT-4 within the IP domain.

应用大型语言模型在知识产权领域具有挑战性，本研究提出了一种低成本、标准化的过程来训练面向知识产权的语言模型，成功满足了该领域的需求，并证明了在知识产权领域中专门训练的模型的有效性。