Despite the recent advancements of attention-based deep learning
architectures across a majority of Natural Language Processing tasks, their
application remains limited in a low-resource setting because of a lack of
pre-trained models for such languages. In this study, we make the first attempt
to investigate the challenges of adapting these techniques for an extremely
low-resource language -- Sumerian cuneiform -- one of the world's oldest
written languages attested from at least the beginning of the 3rd millennium
BC. Specifically, we introduce the first cross-lingual information extraction
pipeline for Sumerian, which includes part-of-speech tagging, named entity
recognition, and machine translation. We further curate InterpretLR, an
interpretability toolkit for low-resource NLP, and use it alongside human
attributions to make sense of the models. We emphasize on human evaluations to
gauge all our techniques. Notably, most components of our pipeline can be
generalised to any other language to obtain an interpretable execution of the
techniques, especially in a low-resource setting. We publicly release all
software, model checkpoints, and a novel dataset with domain-specific
pre-processing to promote further research.

本研究旨在探究注意力深度学习在资源稀缺语言 —— 苏美尔楔形文字中的应用挑战，并介绍了第一个跨语言信息抽取方法，其中包括词性标注、实体识别和机器翻译。同时，开发了一个用于低资源自然语言处理的解释性工具，并重点评估人工干预。我们的方法适用于其他语言，并公开了所有软件、模型和数据集。