Legal professionals need to write analyses that rely on citations to relevant
precedents, i.e., previous case decisions. Intelligent systems assisting legal
professionals in writing such documents provide great benefits but are
challenging to design. Such systems need to help locate, summarize, and reason
over salient precedents in order to be useful. To enable systems for such
tasks, we work with legal professionals to transform a large open-source legal
corpus into a dataset supporting two important backbone tasks: information
retrieval (IR) and retrieval-augmented generation (RAG). This dataset CLERC
(Case Law Evaluation Retrieval Corpus), is constructed for training and
evaluating models on their ability to (1) find corresponding citations for a
given piece of legal analysis and to (2) compile the text of these citations
(as well as previous context) into a cogent analysis that supports a reasoning
goal. We benchmark state-of-the-art models on CLERC, showing that current
approaches still struggle: GPT-4o generates analyses with the highest ROUGE
F-scores but hallucinates the most, while zero-shot IR models only achieve
48.3% recall@1000.

通过转化一个大型开源法律语料库为数据集，我们构建了 CLERC（Case Law Evaluation Retrieval Corpus）数据集，用于培训和评估智能系统在寻找相关引用和编写支持推理目标的连贯分析方面的能力。我们在 CLERC 上对最先进的模型进行了基准测试，结果表明目前的方法仍然存在困难。

CLERC: 法律案件检索数据集及检索增强分析生成

CLERC: A Dataset for Legal Case Retrieval and Retrieval-Augmented  Analysis Generation

In common law jurisdictions, legal practitioners rely on precedents to
construct arguments, in line with the doctrine of \emph{stare decisis}. As the
number of cases grow over the years, prior case retrieval (PCR) has garnered
significant attention. Besides lacking real-world scale, existing PCR datasets
do not simulate a realistic setting, because their queries use complete case
documents while only masking references to prior cases. The query is thereby
exposed to legal reasoning not yet available when constructing an argument for
an undecided case as well as spurious patterns left behind by citation masks,
potentially short-circuiting a comprehensive understanding of case facts and
legal principles. To address these limitations, we introduce a PCR dataset
based on judgements from the European Court of Human Rights (ECtHR), which
explicitly separate facts from arguments and exhibit precedential practices,
aiding us to develop this PCR dataset to foster systems' comprehensive
understanding. We benchmark different lexical and dense retrieval approaches
with various negative sampling strategies, adapting them to deal with long text
sequences using hierarchical variants. We found that difficulty-based negative
sampling strategies were not effective for the PCR task, highlighting the need
for investigation into domain-specific difficulty criteria. Furthermore, we
observe performance of the dense models degrade with time and calls for further
research into temporal adaptation of retrieval models. Additionally, we assess
the influence of different views , Halsbury's and Goodhart's, in practice in
ECtHR jurisdiction using PCR task.

基于欧洲人权法院的判决，推出了一种先例检索（PCR）数据集，以帮助系统全面理解法律事实和原则。通过采用不同的文本检索方法和负采样策略，进一步研究了 PCR 任务的困难度，以及与时间相关的检索模型的性能衰减。此外，还评估了 Halsbury's 和 Goodhart's 在 ECtHR 司法界的不同观点对 PCR 任务的影响。

欧洲人权法庭先例理解与先前案例检索数据集

ECtHR-PCR: A Dataset for Precedent Understanding and Prior Case  Retrieval in the European Court of Human Rights

Legal Judgment Prediction (LJP) has become an increasingly crucial task in
Legal AI, i.e., predicting the judgment of the case in terms of case fact
description. Precedents are the previous legal cases with similar facts, which
are the basis for the judgment of the subsequent case in national legal
systems. Thus, it is worthwhile to explore the utilization of precedents in the
LJP. Recent advances in deep learning have enabled a variety of techniques to
be used to solve the LJP task. These can be broken down into two categories:
large language models (LLMs) and domain-specific models. LLMs are capable of
interpreting and generating complex natural language, while domain models are
efficient in learning task-specific information. In this paper, we propose the
precedent-enhanced LJP framework (PLJP), a system that leverages the strength
of both LLM and domain models in the context of precedents. Specifically, the
domain models are designed to provide candidate labels and find the proper
precedents efficiently, and the large models will make the final prediction
with an in-context precedents comprehension. Experiments on the real-world
dataset demonstrate the effectiveness of our PLJP. Moreover, our work shows a
promising direction for LLM and domain-model collaboration that can be
generalized to other vertical domains.

利用先例加强的 LJP 框架（PLJP）结合了大型语言模型和领域模型的优势，能够有效地预测案件的判决结果，并展示了 LLM 和领域模型协作的有希望的方向。