Zero-Shot Cross-lingual Transfer (ZS-XLT) utilizes a model trained in a
source language to make predictions in another language, often with a
performance loss. To alleviate this, additional improvements can be achieved
through subsequent adaptation using examples in the target language. In this
paper, we exploit In-Context Tuning (ICT) for One-Shot Cross-lingual transfer
in the classification task by introducing In-Context Cross-lingual Transfer
(IC-XLT). The novel concept involves training a model to learn from context
examples and subsequently adapting it during inference to a target language by
prepending a One-Shot context demonstration in that language. Our results show
that IC-XLT successfully leverages target-language examples to improve the
cross-lingual capabilities of the evaluated mT5 model, outperforming
prompt-based models in the Zero and Few-shot scenarios adapted through
fine-tuning. Moreover, we show that when source-language data is limited, the
fine-tuning framework employed for IC-XLT performs comparably to prompt-based
fine-tuning with significantly more training data in the source language.

通过在分类任务中介绍上下文一次性跨语言转换（IC-XLT），可以通过训练模型学习上下文示例，然后在推断时通过在目标语言中预置一次性上下文演示来适应目标语言，提高评估的 mT5 模型的跨语言能力，超过基于提示的模型在经过微调的零和少样本情景中的表现。此外，当源语言数据有限时，IC-XLT 的微调框架与源语言中具有显著更多训练数据的基于提示的微调具有相当的性能。

通过上下文一次性演示实现适应性跨语言文本分类

Adaptive Cross-lingual Text Classification through In-Context One-Shot  Demonstrations

Research interest in task-oriented dialogs has increased as systems such as
Google Assistant, Alexa and Siri have become ubiquitous in everyday life.
However, the impact of academic research in this area has been limited by the
lack of datasets that realistically capture the wide array of user pain points.
To enable research on some of the more challenging aspects of parsing realistic
conversations, we introduce PRESTO, a public dataset of over 550K contextual
multilingual conversations between humans and virtual assistants. PRESTO
contains a diverse array of challenges that occur in real-world NLU tasks such
as disfluencies, code-switching, and revisions. It is the only large scale
human generated conversational parsing dataset that provides structured context
such as a user's contacts and lists for each example. Our mT5 model based
baselines demonstrate that the conversational phenomenon present in PRESTO are
challenging to model, which is further pronounced in a low-resource setup.

本研究介绍了 PRESTO，这是一个基于 550K 个虚拟助手和人之间的上下文多语言对话的公共数据集，用于研究语言理解的挑战性问题，包括流畅性、代码切换和修订等。mT5 模型在该数据集上的表现证明了其中的语言现象具有挑战性，尤其在低资源情况下。

PRESTO: 一个用于解析现实任务导向对话的多语言数据集

PRESTO: A Multilingual Dataset for Parsing Realistic Task-Oriented Dialogs

Indonesian language is spoken by almost 200 million people and is the 10th
most spoken language in the world, but it is under-represented in NLP (Natural
Language Processing) research. A sparsity of language resources has hampered
previous work on Indonesian. The Transformer is a new architecture rapidly
becoming dominant for NLP, surpassing alternatives like convolutional and
recurrent neural networks. T5 (Text-to-Text Transfer Transformer) is a
Transformer model that converts all text-based language problems to
text-to-text format for English. The multilingual variant is mT5 (multilingual
T5) which has shown promising results on many NLP tasks across languages.
However, the size of this multilingual model is a drawback for its application
in real production applications, which sometimes require only one language. In
this study, the mT5 model was adapted for only one language, Indonesian,
resulting in a pre-trained T5 model that was specific only for Indonesian with
a smaller size. For performance comparison, we fine-tuned this model and the
mT5 model to the Sentiment Analysis (SA), Question Generation (QG), and
Question Answering (QA) tasks with the exact mechanism and dataset. Fine-tuned
model based on our model achieved 77.18% accuracy on SA, 8% higher than the
mT5-based model, and obtained nearly the same score as the mT5-based model on
QG and QA. The results confirm that it is possible to produce a smaller
pre-trained model that maintains comparable yields while reducing the model
size by up to 58%. In addition, the resulting model requires less memory, loads
faster, and inference times faster.

本文研究了将 mT5 多语言模型适应为仅针对印尼语的预训练 T5 模型，通过对多语言模型与本模型在情感分析、问题生成和问答三个任务的表现进行比较，结果表明可以生成一个更小的预训练模型并在 comparable yields 的同时减小模型大小高达 58％，而且所得模型需要更少的内存，加载速度更快，推理时间更快。