The BLOOM model is a large open-source multilingual language model capable of
zero-shot learning, but its pretraining was limited to 46 languages. To improve
its zero-shot performance on unseen languages, it is desirable to adapt BLOOM,
but previous works have only explored adapting small language models. In this
work, we apply existing language adaptation strategies to BLOOM and benchmark
its zero-shot prompting performance on eight new languages. We find language
adaptation to be effective at improving zero-shot performance in new languages.
Surprisingly, adapter-based finetuning is more effective than continued
pretraining for large models. In addition, we discover that prompting
performance is not significantly affected by language specifics, such as the
writing system. It is primarily determined by the size of the language
adaptation data. We also add new languages to BLOOMZ, which is a multitask
finetuned version of BLOOM capable of following task instructions zero-shot. We
find including a new language in the multitask fine-tuning mixture to be the
most effective method to teach BLOOMZ a new language. We conclude that with
sufficient training data language adaptation can generalize well to diverse
languages. Our code is available at
https://github.com/bigscience-workshop/multilingual-modeling/.

该研究旨在提高 BLOOM 模型在未见过语言上的度量表现。研究发现，通过进行语言适应性策略来提高它的零样本度量表现是有效的，快速微调适配器比继续预训练更有效，语言度量表现主要取决于适应数据的大小。同时，通过将新语言添加到多任务微调混合中可以有效地教授 BLOOMZ 模型一种新语言。

BLOOM+1: BLOOM 对零 - shot 提示增加语言支持

BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting

Despite achieving state-of-the-art zero-shot performance, existing
vision-language models still fall short of few-shot transfer ability on
domain-specific problems. Classical fine-tuning often fails to prevent highly
expressive models from exploiting spurious correlations. Although
model-agnostic meta-learning (MAML) presents as a natural alternative for
few-shot transfer learning, the expensive computation due to implicit
second-order optimization limits its use on large-scale vision-language models
such as CLIP. While much literature has been devoted to exploring alternative
optimization strategies, we identify another essential aspect towards effective
few-shot transfer learning, task sampling, which is previously only be viewed
as part of data pre-processing in MAML. To show the impact of task sampling, we
propose a simple algorithm, Model-Agnostic Multitask Fine-tuning (MAMF), which
differentiates classical fine-tuning only on uniformly sampling multiple tasks.
Despite its simplicity, we show that MAMF consistently outperforms classical
fine-tuning on five few-shot vision-language classification tasks. We further
show that the effectiveness of the bi-level optimization in MAML is highly
sensitive to the zero-shot performance of a task in the context of few-shot
vision-language classification. The goal of this paper is to provide new
insights on what makes few-shot learning work, and encourage more research into
investigating better task sampling strategies.

提出了一种新的模型无关多任务微调算法（MAMF）并探究了任务抽样对于有效进行少样本学习的影响，表明 MAMF 在五个少样本视觉语言分类任务上表现比经典的微调更佳，旨在为少样本学习提供新的见解，并鼓励探究更好的任务抽样策略。