In this paper, we explore the application of Generative Pre-trained
Transformers (GPTs) in cross-lingual legal Question-Answering (QA) systems
using the COLIEE Task 4 dataset. In the COLIEE Task 4, given a statement and a
set of related legal articles that serve as context, the objective is to
determine whether the statement is legally valid, i.e., if it can be inferred
from the provided contextual articles or not, which is also known as an
entailment task. By benchmarking four different combinations of English and
Japanese prompts and data, we provide valuable insights into GPTs' performance
in multilingual legal QA scenarios, contributing to the development of more
efficient and accurate cross-lingual QA solutions in the legal domain.

利用 COLIEE 任务 4 数据集，探索生成式预训练转换器（GPTs）在跨语言法律问答系统中的应用，通过对四种不同的英语和日语提示和数据进行基准测试，为多语言法律问答方案的开发提供有价值的洞察，以提高效率和准确性。

GPT 和语言障碍：跨语言法律问答考察

GPTs and Language Barrier: A Cross-Lingual Legal QA Examination

The evolution of Generative Pre-trained Transformer (GPT) models has led to
significant advancements in various natural language processing applications,
particularly in legal textual entailment. We present an analysis of GPT-3.5
(ChatGPT) and GPT-4 performances on COLIEE Task 4 dataset, a prominent
benchmark in this domain. The study encompasses data from Heisei 18 (2006) to
Reiwa 3 (2021), exploring the models' abilities to discern entailment
relationships within Japanese statute law across different periods. Our
preliminary experimental results unveil intriguing insights into the models'
strengths and weaknesses in handling legal textual entailment tasks, as well as
the patterns observed in model performance. In the context of proprietary
models with undisclosed architectures and weights, black-box analysis becomes
crucial for evaluating their capabilities. We discuss the influence of training
data distribution and the implications on the models' generalizability. This
analysis serves as a foundation for future research, aiming to optimize
GPT-based models and enable their successful adoption in legal information
extraction and entailment applications.

对 COLIEE 任务 4 数据集中 Heisei 18（2006 年）到 Reiwa 3（2021 年）的日本法律文本具体判断能力的 GPT-3.5（ChatGPT）和 GPT-4 模型的分析揭示了模型在处理法律文本具体性任务方面的优点、缺点和性能模式，为未来优化 GPT-based 模型并在法律信息提取和具体性应用中成功采用奠定了基础。