Historical languages present unique challenges to the NLP community, with one
prominent hurdle being the limited resources available in their closed corpora.
This work describes our submission to the constrained subtask of the SIGTYP
2024 shared task, focusing on PoS tagging, morphological tagging, and
lemmatization for 13 historical languages. For PoS and morphological tagging we
adapt a hierarchical tokenization method from Sun et al. (2023) and combine it
with the advantages of the DeBERTa-V3 architecture, enabling our models to
efficiently learn from every character in the training data. We also
demonstrate the effectiveness of character-level T5 models on the lemmatization
task. Pre-trained from scratch with limited data, our models achieved first
place in the constrained subtask, nearly reaching the performance levels of the
unconstrained task's winner. Our code is available at
this https URL

历史语言中的 NLP 社区所面临的主要挑战之一是其封闭语料库中有限的资源。本研究描述了我们参与 SIGTYP 2024 共享任务约束子任务的提交，重点关注 13 种历史语言的词性标注、形态标注和词形还原。我们采用了 Sun 等人（2023 年）的分层分词方法，并结合 DeBERTa-V3 架构的优势，使我们的模型能够有效地从训练数据的每个字符中学习。我们还展示了字符级 T5 模型在词形还原任务中的有效性。我们的模型通过有限的数据从头开始预训练，并在约束子任务中获得了第一名，几乎达到了无约束任务的冠军水平。我们的代码可在此 https URL 找到。

海德堡 - 波士顿 @ SIGTYP 2024 共享任务：使用字符感知分层变形器增强低资源语言分析

Heidelberg-Boston @ SIGTYP 2024 Shared Task: Enhancing Low-Resource  Language Analysis With Character-Aware Hierarchical Transformers

In this work, we explore whether the recently demonstrated zero-shot
abilities of the T0 model extend to Named Entity Recognition for
out-of-distribution languages and time periods. Using a historical newspaper
corpus in 3 languages as test-bed, we use prompts to extract possible named
entities. Our results show that a naive approach for prompt-based zero-shot
multilingual Named Entity Recognition is error-prone, but highlights the
potential of such an approach for historical languages lacking labeled
datasets. Moreover, we also find that T0-like models can be probed to predict
the publication date and language of a document, which could be very relevant
for the study of historical texts.

本文探讨了使用 T0 模型的 zero-shot 技术来进行跨越时间和语言的 NER 是否可行，通过历史文献的测试，结果表明利用 prompt 的 naive 方法在 zero-shot 多语种 NER 方面存在误差，但揭示了该方法在缺乏标记数据的历史语言方面的潜力。此外，实验结果也表明类似 T0 的模型可以用于预测文档的发布日期和语言，这对于历史文本研究非常有意义。