Court transcripts and judgments are rich repositories of legal knowledge,
detailing the intricacies of cases and the rationale behind judicial decisions.
The extraction of key information from these documents provides a concise
overview of a case, crucial for both legal experts and the public. With the
advent of large language models (LLMs), automatic information extraction has
become increasingly feasible and efficient. This paper presents a comprehensive
study on the application of GPT-4, a large language model, for automatic
information extraction from UK Employment Tribunal (UKET) cases. We
meticulously evaluated GPT-4's performance in extracting critical information
with a manual verification process to ensure the accuracy and relevance of the
extracted data. Our research is structured around two primary extraction tasks:
the first involves a general extraction of eight key aspects that hold
significance for both legal specialists and the general public, including the
facts of the case, the claims made, references to legal statutes, references to
precedents, general case outcomes and corresponding labels, detailed order and
remedies and reasons for the decision. The second task is more focused, aimed
at analysing three of those extracted features, namely facts, claims and
outcomes, in order to facilitate the development of a tool capable of
predicting the outcome of employment law disputes. Through our analysis, we
demonstrate that LLMs like GPT-4 can obtain high accuracy in legal information
extraction, highlighting the potential of LLMs in revolutionising the way legal
information is processed and utilised, offering significant implications for
legal research and practice.

这篇论文通过对英国雇佣法庭（UKET）案件中使用 GPT-4 进行自动信息提取的综合研究，展示了语言模型在法律信息提取方面的高精度和潜力，并对工具开发提供了指导。

使用大型语言模型自动提取雇佣仲裁判决中的信息

Automatic Information Extraction From Employment Tribunal Judgements  Using Large Language Models

As humans, we consistently engage in interactions with our peers and receive
feedback in the form of natural language. This language feedback allows us to
reflect on our actions, maintain appropriate behavior, and rectify our errors.
The question arises naturally: can we use language feedback to align large
language models (LLMs)? In contrast to previous research that aligns LLMs with
reward or preference data, we present the first systematic exploration of
alignment through the lens of language feedback (i.e., judgment). We commence
with an in-depth investigation of potential methods that can be adapted for
aligning LLMs with judgments, revealing that these methods are unable to fully
capitalize on the judgments. To facilitate more effective utilization of
judgments, we propose a novel framework, Contrastive Unlikelihood Training
(CUT), that allows for fine-grained inappropriate content detection and
correction based on judgments. Our offline alignment results show that, with
merely 1317 off-the-shelf judgment data, CUT (LLaMA2-13b) can beat the 175B
DaVinci003 and surpass the best baseline by 52.34 points on AlpacaEval. The
online alignment results demonstrate that CUT can align LLMs (LLaMA2-chat-13b)
in an iterative fashion using model-specific judgment data, with a steady
performance improvement from 81.09 to 91.36 points on AlpacaEval. Our analysis
further suggests that judgments exhibit greater potential than rewards for LLM
alignment and warrant future research.

我们首次通过自然语言反馈的方法探索了对齐大型语言模型的可能性，并提出了一种称为 Contrastive Unlikelihood Training (CUT) 的新框架，通过细致判定检测和修正来实现对不适当内容的改进，获得了优于基线模型的好成绩。同时，我们的分析表明判定相较于奖励在 LLM 对齐方面具有更大的潜力，值得进行进一步研究。

拒绝的原因？将语言模型与判断对齐

Reasons to Reject? Aligning Language Models with Judgments

Developing methods for extracting relevant legal information to aid legal
practitioners is an active research area. In this regard, research efforts are
being made by leveraging different kinds of information, such as meta-data,
citations, keywords, sentences, paragraphs, etc. Similar to any text document,
legal documents are composed of paragraphs. In this paper, we have analyzed the
resourcefulness of paragraph-level information in capturing similarity among
judgments for improving the performance of precedence retrieval. We found that
the paragraph-level methods could capture the similarity among the judgments
with only a few paragraph interactions and exhibit more discriminating power
over the baseline document-level method. Moreover, the comparison results on
two benchmark datasets for the precedence retrieval on the Indian supreme court
judgments task show that the paragraph-level methods exhibit comparable
performance with the state-of-the-art methods

通过对法律文件进行段落级信息分析，可以提高先例检索的性能，并且与现有的方法相比表现出可比较的性能。