This research aims to build generative language models specialized for the
legal domain. The manuscript presents the development of LexGPT models based on
GPT-J models and pre-trained with Pile of Law. The foundation model built in
this manuscript is the initial step for the development of future applications
in the legal domain, such as further training with reinforcement learning from
human feedback. Another objective of this manuscript is to assist legal
professionals in utilizing language models through the ``No Code'' approach. By
fine-tuning models with specialized data and without modifying any source code,
legal professionals can create custom language models for downstream tasks with
minimum effort and technical knowledge. The downstream task in this manuscript
is to turn a LexGPT model into a classifier, although the performance is
notably lower than the state-of-the-art result. How to enhance downstream task
performance without modifying the model or its source code is a research topic
for future exploration.

该篇论文旨在构建专为法律领域定制的生成语言模型，其中 LexGPT 模型可通过 “无代码” 方法对专业数据进行微调，以用于下游任务，但分类器性能比最先进的结果明显较低。