In this paper, we present PARAMANU-AYN, a language model based exclusively on
case documents of the Supreme Court of India, the Constitution of India, and
the Indian Penal Code. The novel Auto Regressive (AR) decoder based model is
pretrained from scratch at a context size of 8192. We evaluated our pretrained
legal model on perplexity metrics. We also instruction-tuned our pretrained
model on a set of 10,763 instructions covering various legal tasks such as
legal reasoning, judgement explanation, legal clause generation, legal
drafting, legal contract drafting, case summarization, constitutional
question-answering, etc. We also evaluated the responses of prompts for
instruction-tuned models by GPT-3.5-Turbo on clarity, relevance, completeness,
and legal reasoning metrics in a scale of 10. Our model can be run on CPU and
achieved 42.46 tokens/sec CPU inference speed. We found that our models,
despite not being pretrained on legal books, various legal contracts, and legal
documents, were able to learn the domain knowledge required for drafting
various legal contracts and legal clauses, and generalize to draft legal
contracts and legal clauses with limited instruction tuning. Hence, we conclude
that for a strong domain-specialized generative language model (such as legal),
very large amounts of data are not required to develop models from scratch. We
believe that this work is the first attempt to make a dedicated generative
legal language model from scratch for Indian Supreme Court jurisdiction or in
legal NLP overall. We plan to release our Paramanu-Ayn model at
this https URL

本研究介绍了 PARAMANU-AYN，这是一种基于印度最高法院案件文件、印度宪法和印度刑法的语言模型。该模型使用了基于自回归解码器的新型预训练方法，通过训练和评估多个指令任务，证明了即使没有大量数据，也能从头开始开发一个专门的生成式法律语言模型。

PARAMANU-AYN: 适用于印度法律案件文件的高效新型生成和指导语言模型

PARAMANU-AYN: An Efficient Novel Generative and Instruction-tuned  Language Model for Indian Legal Case Documents

Automatically understanding the rhetorical roles of sentences in a legal case
judgement is an important problem to solve, since it can help in several
downstream tasks like summarization of legal judgments, legal search, and so
on. The task is challenging since legal case documents are usually not
well-structured, and these rhetorical roles may be subjective (as evident from
variation of opinions between legal experts). In this paper, we address this
task for judgments from the Supreme Court of India. We label sentences in 50
documents using multiple human annotators, and perform an extensive analysis of
the human-assigned labels. We also attempt automatic identification of the
rhetorical roles of sentences. While prior approaches towards this task used
Conditional Random Fields over manually handcrafted features, we explore the
use of deep neural models which do not require hand-crafting of features.
Experiments show that neural models perform much better in this task than
baseline methods which use handcrafted features.

本文通过使用深度神经网络，解决了自动理解印度最高法院判决中句子的修辞角色的难题，从而达成了对法律案例做出摘要和搜索等下游任务的目的。