Multilingual language models enable zero-shot cross-lingual transfer
(ZS-XLT): fine-tuned on sizable source-language task data, they perform the
task in target languages without labeled instances. The effectiveness of ZS-XLT
hinges on the linguistic proximity between languages and the amount of
pretraining data for a language. Because of this, model selection based on
source-language validation is unreliable: it picks model snapshots with
suboptimal target-language performance. As a remedy, some work optimizes ZS-XLT
by extensively tuning hyperparameters: the follow-up work then routinely
struggles to replicate the original results. Other work searches over narrower
hyperparameter grids, reporting substantially lower performance. In this work,
we therefore propose an unsupervised evaluation protocol for ZS-XLT that
decouples performance maximization from hyperparameter tuning. As a robust and
more transparent alternative to extensive hyperparameter tuning, we propose to
accumulatively average snapshots from different runs into a single model. We
run broad ZS-XLT experiments on both higher-level semantic tasks (NLI,
extractive QA) and a lower-level token classification task (NER) and find that
conventional model selection based on source-language validation quickly
plateaus to suboptimal ZS-XLT performance. On the other hand, our accumulative
run-by-run averaging of models trained with different hyperparameters boosts
ZS-XLT performance and closely correlates with "oracle" ZS-XLT, i.e., model
selection based on target-language validation performance.

基于不同运行模型的累积逐渐平均，提升零样本跨语言转移（ZS-XLT）性能，与目标语言验证性能基于模型选择有密切相关。