To undertake computational research of the law, efficiently identifying
datasets of court decisions that relate to a specific legal issue is a crucial
yet challenging endeavour. This study addresses the gap in the literature
working with large legal corpora about how to isolate cases, in our case
summary judgments, from a large corpus of UK court decisions. We introduce a
comparative analysis of two computational methods: (1) a traditional natural
language processing-based approach leveraging expert-generated keywords and
logical operators and (2) an innovative application of the Claude 2 large
language model to classify cases based on content-specific prompts. We use the
Cambridge Law Corpus of 356,011 UK court decisions and determine that the large
language model achieves a weighted F1 score of 0.94 versus 0.78 for keywords.
Despite iterative refinement, the search logic based on keywords fails to
capture nuances in legal language. We identify and extract 3,102 summary
judgment cases, enabling us to map their distribution across various UK courts
over a temporal span. The paper marks a pioneering step in employing advanced
natural language processing to tackle core legal research tasks, demonstrating
how these technologies can bridge systemic gaps and enhance the accessibility
of legal information. We share the extracted dataset metrics to support further
research on summary judgments.

本文使用自然语言处理技术，比较了基于关键词和逻辑运算符的传统方法与基于 Claude 2 大型语言模型的创新方法，在英国法院裁决案例的大语料库中提取总结性判决案例。结果表明，大语言模型的加权 F1 得分为 0.94，而关键词法的得分仅为 0.78，说明大语言模型在捕捉法律语言中的细微差别方面更加有效。本文展示了先进自然语言处理技术在核心法律研究任务中的应用，并且阐明了这些技术如何填补系统性差距并提升法律信息的可获取性。同时，我们分享了提取的数据集度量，以支持进一步的总结性判决研究。

LLM 与律师：在大型英国案例法律数据库中识别摘要判决的子集

LLM vs. Lawyers: Identifying a Subset of Summary Judgments in a Large UK  Case Law Dataset

Developing methods for extracting relevant legal information to aid legal
practitioners is an active research area. In this regard, research efforts are
being made by leveraging different kinds of information, such as meta-data,
citations, keywords, sentences, paragraphs, etc. Similar to any text document,
legal documents are composed of paragraphs. In this paper, we have analyzed the
resourcefulness of paragraph-level information in capturing similarity among
judgments for improving the performance of precedence retrieval. We found that
the paragraph-level methods could capture the similarity among the judgments
with only a few paragraph interactions and exhibit more discriminating power
over the baseline document-level method. Moreover, the comparison results on
two benchmark datasets for the precedence retrieval on the Indian supreme court
judgments task show that the paragraph-level methods exhibit comparable
performance with the state-of-the-art methods

通过对法律文件进行段落级信息分析，可以提高先例检索的性能，并且与现有的方法相比表现出可比较的性能。