Interpreting the meaning of legal open-textured terms is a key task of legal
professionals. An important source for this interpretation is how the term was
applied in previous court cases. In this paper, we evaluate the performance of
GPT-4 in generating factually accurate, clear and relevant explanations of
terms in legislation. We compare the performance of a baseline setup, where
GPT-4 is directly asked to explain a legal term, to an augmented approach,
where a legal information retrieval module is used to provide relevant context
to the model, in the form of sentences from case law. We found that the direct
application of GPT-4 yields explanations that appear to be of very high quality
on their surface. However, detailed analysis uncovered limitations in terms of
the factual accuracy of the explanations. Further, we found that the
augmentation leads to improved quality, and appears to eliminate the issue of
hallucination, where models invent incorrect statements. These findings open
the door to the building of systems that can autonomously retrieve relevant
sentences from case law and condense them into a useful explanation for legal
scholars, educators or practicing lawyers alike.

对 GPT-4 在生成有关立法术语方面的准确性、清晰性和相关性上的表现进行评估。与基准设置相比，采用法律信息检索模块提供以前案例中的语句作为模型的上下文可以获得更好的结果，从而可以自主检索相关案例，并将这些案例中有用的语句压缩成有用的法律解释。

增强的大型语言模型（GPT-4）用于解释法律概念

Explaining Legal Concepts with Augmented Large Language Models (GPT-4)

Case law retrieval is the retrieval of judicial decisions relevant to a legal
question. Case law retrieval comprises a significant amount of a lawyer's time,
and is important to ensure accurate advice and reduce workload. We survey
methods for case law retrieval from the past 20 years and outline the problems
and challenges facing evaluation of case law retrieval systems going forward.
Limited published work has focused on improving ranking in ad-hoc case law
retrieval. But there has been significant work in other areas of case law
retrieval, and legal information retrieval generally. This is likely due to
legal search providers being unwilling to give up the secrets of their success
to competitors. Most evaluations of case law retrieval have been undertaken on
small collections and focus on related tasks such as question-answer systems or
recommender systems. Work has not focused on Cranfield style evaluations and
baselines of methods for case law retrieval on publicly available test
collections are not present. This presents a major challenge going forward. But
there are reasons to question the extent of this problem, at least in a
commercial setting. Without test collections to baseline approaches it cannot
be known whether methods are promising. Works by commercial legal search
providers show the effectiveness of natural language systems as well as query
expansion for case law retrieval. Machine learning is being applied to more and
more legal search tasks, and undoubtedly this represents the future of case law
retrieval.

研究案例检索的问题及挑战，介绍过去 20 年的检索方法，指出了评估案例检索系统的问题，并讨论了机器学习在案例检索中的应用前景。

过去 20 年中判例检索的问题、方法、挑战和评估

Case law retrieval: problems, methods, challenges and evaluations in the last 20 years

This paper presents a study of employing Ranking SVM and Convolutional Neural
Network for two missions: legal information retrieval and question answering in
the Competition on Legal Information Extraction/Entailment. For the first task,
our proposed model used a triple of features (LSI, Manhattan, Jaccard), and is
based on paragraph level instead of article level as in previous studies. In
fact, each single-paragraph article corresponds to a particular paragraph in a
huge multiple-paragraph article. For the legal question answering task,
additional statistical features from information retrieval task integrated into
Convolutional Neural Network contribute to higher accuracy.

该研究采用 Ranking SVM 和卷积神经网络进行法律信息检索和问答，并通过信息检索任务中的统计特征提高了卷积神经网络的准确性。