Large Language Models (LLMs) have shown the potential to revolutionize
natural language processing tasks in various domains, sparking great interest
in vertical-specific large models. However, unlike proprietary models such as
BloombergGPT and FinGPT, which have leveraged their unique data accumulations
to make strides in the finance domain, there hasn't not many similar large
language models in the Chinese legal domain to facilitate its digital
transformation.
In this paper, we propose an open-source legal large language model named
ChatLaw. Due to the importance of data quality, we carefully designed a legal
domain fine-tuning dataset. Additionally, to overcome the problem of model
hallucinations in legal data screening during reference data retrieval, we
introduce a method that combines vector database retrieval with keyword
retrieval to effectively reduce the inaccuracy of relying solely on vector
database retrieval. Furthermore, we propose a self-attention method to enhance
the ability of large models to overcome errors present in reference data,
further optimizing the issue of model hallucinations at the model level and
improving the problem-solving capabilities of large models. We also
open-sourced our model and part of the data at
this https URL

本文介绍了一个名为 ChatLaw 的开源法律大语言模型，它通过特定的数据集、关键词检索和自注意力机制来优化模型在法律领域的表现，解决了参考数据筛选过程中的模型幻觉问题，提升了其解决问题的能力。