Prompt-based methods have been successfully applied to multilingual
pretrained language models for zero-shot cross-lingual understanding. However,
most previous studies primarily focused on sentence-level classification tasks,
and only a few considered token-level labeling tasks such as Named Entity
Recognition (NER) and Part-of-Speech (POS) tagging. In this paper, we propose
Token-Level Prompt Decomposition (ToPro), which facilitates the prompt-based
method for token-level sequence labeling tasks. The ToPro method decomposes an
input sentence into single tokens and applies one prompt template to each
token. Our experiments on multilingual NER and POS tagging datasets demonstrate
that ToPro-based fine-tuning outperforms Vanilla fine-tuning and Prompt-Tuning
in zero-shot cross-lingual transfer, especially for languages that are
typologically different from the source language English. Our method also
attains state-of-the-art performance when employed with the mT5 model. Besides,
our exploratory study in multilingual large language models shows that ToPro
performs much better than the current in-context learning method. Overall, the
performance improvements show that ToPro could potentially serve as a novel and
simple benchmarking method for sequence labeling tasks.

通过将输入句子分解为单个标记并对每个标记应用一个提示模板，我们提出了一种名为 ToPro 的基于提示的方法，用于标记任务的标记级序列标注。在多语言 NER 和 POS 标记数据集上的实验证明，与 Vanilla fine-tuning 和 Prompt-Tuning 相比，ToPro-based fine-tuning 在零样本跨语言迁移方面表现优秀，尤其对于与源语言英语在语言类型上有所不同的语言。此外，我们在多语言大语言模型上的初步研究表明，ToPro 的性能要远远优于当前的上下文学习方法。总体而言，性能的提升表明 ToPro 有望成为标记任务的一种新颖且简单的基准方法。