Automatic legal judgment prediction and its explanation suffer from the
problem of long case documents exceeding tens of thousands of words, in
general, and having a non-uniform structure. Predicting judgments from such
documents and extracting their explanation becomes a challenging task, more so
on documents with no structural annotation. We define this problem as "scarce
annotated legal documents" and explore their lack of structural information and
their long lengths with a deep learning-based classification framework which we
call MESc; "Multi-stage Encoder-based Supervised with-clustering"; for judgment
prediction. Specifically, we divide a document into parts to extract their
embeddings from the last four layers of a custom fine-tuned Large Language
Model, and try to approximate their structure through unsupervised clustering.
Which we use in another set of transformer encoder layers to learn the
inter-chunk representations. We explore the adaptability of LLMs with
multi-billion parameters (GPT-Neo, and GPT-J) to legal texts and their
intra-domain(legal) transfer learning capacity. Alongside this, we compare
their performance with MESc and the impact of combining embeddings from their
last layers. For such hierarchical models, we also propose an explanation
extraction algorithm named ORSE; Occlusion sensitivity-based Relevant Sentence
Extractor;

对于长达数万字、结构不均匀的案例文件，自动法律判决预测及其解释存在困难。本研究使用称为 MESc 的深度学习分类框架，通过定义 “缺少结构化信息的法律文档” 问题，尝试从这些缺少结构标注的长文档中提取解释。同时，我们还研究了拥有数十亿参数的大型语言模型（GPT-Neo 和 GPT-J）在法律文本和领域内迁移学习方面的适应性，并将其性能与 MESc 及其最后层嵌入的组合进行了比较。对于这种层次模型，我们提出了一种名为 ORSE 的解释提取算法，即基于遮蔽敏感性的相关句子提取器。