We present DALE, a novel and effective generative Data Augmentation framework
for low-resource LEgal NLP. DALE addresses the challenges existing frameworks
pose in generating effective data augmentations of legal documents - legal
language, with its specialized vocabulary and complex semantics, morphology,
and syntax, does not benefit from data augmentations that merely rephrase the
source sentence. To address this, DALE, built on an Encoder-Decoder Language
Model, is pre-trained on a novel unsupervised text denoising objective based on
selective masking - our masking strategy exploits the domain-specific language
characteristics of templatized legal documents to mask collocated spans of
text. Denoising these spans helps DALE acquire knowledge about legal concepts,
principles, and language usage. Consequently, it develops the ability to
generate coherent and diverse augmentations with novel contexts. Finally, DALE
performs conditional generation to generate synthetic augmentations for
low-resource Legal NLP tasks. We demonstrate the effectiveness of DALE on 13
datasets spanning 6 tasks and 4 low-resource settings. DALE outperforms all our
baselines, including LLMs, qualitatively and quantitatively, with improvements
of 1%-50%.

DALE 是一个新颖有效的低资源法律 NLP 数据增强框架，通过使用编码 - 解码语言模型，预训练选择性屏蔽的无监督文本去噪目标，结合特定领域语言特性对模板化法律文件进行遮盖，从而生成连贯多样的合成增强，用于低资源法律 NLP 任务。DALE 在 13 个数据集上展示了其优越性能，在质量和数量上都优于 LLMs 等其他基线方法，改进幅度在 1%-50% 之间。