Legal practice has witnessed a sharp rise in products incorporating
artificial intelligence (AI). Such tools are designed to assist with a wide
range of core legal tasks, from search and summarization of caselaw to document
drafting. But the large language models used in these tools are prone to
"hallucinate," or make up false information, making their use risky in
high-stakes domains. Recently, certain legal research providers have touted
methods such as retrieval-augmented generation (RAG) as "eliminating"
(Casetext, 2023) or "avoid[ing]" hallucinations (Thomson Reuters, 2023), or
guaranteeing "hallucination-free" legal citations (LexisNexis, 2023). Because
of the closed nature of these systems, systematically assessing these claims is
challenging. In this article, we design and report on the first preregistered
empirical evaluation of AI-driven legal research tools. We demonstrate that the
providers' claims are overstated. While hallucinations are reduced relative to
general-purpose chatbots (GPT-4), we find that the AI research tools made by
LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and
Ask Practical Law AI) each hallucinate between 17% and 33% of the time. We also
document substantial differences between systems in responsiveness and
accuracy. Our article makes four key contributions. It is the first to assess
and report the performance of RAG-based proprietary legal AI tools. Second, it
introduces a comprehensive, preregistered dataset for identifying and
understanding vulnerabilities in these systems. Third, it proposes a clear
typology for differentiating between hallucinations and accurate legal
responses. Last, it provides evidence to inform the responsibilities of legal
professionals in supervising and verifying AI outputs, which remains a central
open question for the responsible integration of AI into law.

人工智能在法律实践中的应用产生了假象问题，本文通过评估 AI 驱动的法律研究工具，发现这些工具存在假象问题，并提出了区分假象和准确法律回答的方法，为法律专业人士监督和验证人工智能输出提供了证据。

能否无幻觉？评估主流人工智能法律研究工具的可靠性

Hallucination-Free? Assessing the Reliability of Leading AI Legal  Research Tools

The current fascination with large language models, or LLMs, derives from the
fact that many users lack the expertise to evaluate the quality of the
generated text. LLMs may therefore appear more capable than they actually are.
The dangerous combination of fluency and superficial plausibility leads to the
temptation to trust the generated text and creates the risk of overreliance.
Who would not trust perfect legalese? Relying recent findings in both technical
and legal scholarship, this Article counterbalances the overly optimistic
predictions as to the role of LLMs in legal practice. Integrating LLMs into
legal workstreams without a better comprehension of their limitations, will
create inefficiencies if not outright risks. Notwithstanding their
unprecedented ability to generate text, LLMs do not understand text. Without
the ability to understand meaning, LLMs will remain unable to use language, to
acquire knowledge and to perform complex reasoning tasks. Trained to model
language on the basis of stochastic word predictions, LLMs cannot distinguish
fact from fiction. Their knowledge of the law is limited to word strings
memorized in their parameters. It is also incomplete and largely incorrect.
LLMs operate at the level of word distributions, not at the level of verified
facts. The resulting propensity to hallucinate, to produce statements that are
incorrect but appear helpful and relevant, is alarming in high-risk areas like
legal services. At present, lawyers should beware of relying on text generated
by LLMs.

本文通过结合技术和法律学术的最新发现，对大型语言模型（LLMs）在法律实践中的作用和限制进行了反过来的评估，指出过度乐观的预测将导致 LLMs 在法律领域引发低效和风险。

阅读注意：大型语言模型在法律实践中的应用

Caveat Lector: Large Language Models in Legal Practice

Legal practice is intrinsically rooted in the fabric of language, yet legal
practitioners and scholars have been slow to adopt tools from natural language
processing (NLP). At the same time, the legal system is experiencing an access
to justice crisis, which could be partially alleviated with NLP. In this
position paper, we argue that the slow uptake of NLP in legal practice is
exacerbated by a disconnect between the needs of the legal community and the
focus of NLP researchers. In a review of recent trends in the legal NLP
literature, we find limited overlap between the legal NLP community and legal
academia. Our interpretation is that some of the most popular legal NLP tasks
fail to address the needs of legal practitioners. We discuss examples of legal
NLP tasks that promise to bridge disciplinary disconnects and highlight
interesting areas for legal NLP research that remain underexplored.

通过对最近法律自然语言处理（NLP）文献中的趋势的回顾，我们发现法律 NLP 社区与法学界之间的重叠有限。我们认为，一些最流行的法律 NLP 任务未能解决法律从业人员的需求，因此我们探讨了一些能够弥合学科鸿沟的法律 NLP 任务示例，并突出了一些尚未充分探索的有趣领域。