In the intricate field of legal studies, the analysis of court decisions is a
cornerstone for the effective functioning of the judicial system. The ability
to predict court outcomes helps judges during the decision-making process and
equips lawyers with invaluable insights, enhancing their strategic approaches
to cases. Despite its significance, the domain of Arabic court analysis remains
under-explored. This paper pioneers a comprehensive predictive analysis of
Arabic court decisions on a dataset of 10,813 commercial court real cases,
leveraging the advanced capabilities of the current state-of-the-art large
language models. Through a systematic exploration, we evaluate three prevalent
foundational models (LLaMA-7b, JAIS-13b, and GPT3.5-turbo) and three training
paradigms: zero-shot, one-shot, and tailored fine-tuning. Besides, we assess
the benefit of summarizing and/or translating the original Arabic input texts.
This leads to a spectrum of 14 model variants, for which we offer a granular
performance assessment with a series of different metrics (human assessment,
GPT evaluation, ROUGE, and BLEU scores). We show that all variants of LLaMA
models yield limited performance, whereas GPT-3.5-based models outperform all
other models by a wide margin, surpassing the average score of the dedicated
Arabic-centric JAIS model by 50%. Furthermore, we show that all scores except
human evaluation are inconsistent and unreliable for assessing the performance
of large language models on court decision predictions. This study paves the
way for future research, bridging the gap between computational linguistics and
Arabic legal analytics.

在阿拉伯法律分析领域中，我们的研究通过使用现有最先进的大型语言模型，全面预测了一批基于商业法庭真实案例的阿拉伯法院判决。通过评估不同预训练模型和训练方法的性能，并采用不同的评估指标，我们发现基于 GPT-3.5 的模型在所有模型中表现最佳，超过专门针对阿拉伯文的 JAIS 模型平均得分的 50%。同时，我们还发现大语言模型在法院判决预测中的性能评估除了人工评估之外的其他指标都是不一致且不可靠的。本研究为未来的研究奠定了基础，弥合计算语言学与阿拉伯法律分析之间的差距。