Transferring learned patterns from pretrained neural language models has been
shown to significantly improve effectiveness across a variety of language-based
tasks, meanwhile further tuning on intermediate tasks has been demonstrated to
provide additional performance benefits, provided the intermediate task is
sufficiently related to the target task. However, how to identify related tasks
is an open problem, and brute-force searching effective task combinations is
prohibitively expensive. Hence, the question arises, are we able to improve the
effectiveness and efficiency of tasks with no training examples through
selective fine-tuning? In this paper, we explore statistical measures that
approximate the divergence between domain representations as a means to
estimate whether tuning using one task pair will exhibit performance benefits
over tuning another. This estimation can then be used to reduce the number of
task pairs that need to be tested by eliminating pairs that are unlikely to
provide benefits. Through experimentation over 58 tasks and over 6,600 task
pair combinations, we demonstrate that statistical measures can distinguish
effective task pairs, and the resulting estimates can reduce end-to-end runtime
by up to 40%.

本文探讨使用统计量来衡量不同任务间的相似性，从而减少 fine-tuning 的测试所需的任务对数量，并在 58 个任务和 6600 多个任务对组合上实验证明其可以提高任务的效率并显著降低运行时间。