Multi-task learning can leverage information learned by one task to benefit
the training of other tasks. Despite this capacity, naive formulations often
degrade performance and in particular, identifying the tasks that would benefit
from co-training remains a challenging design question. In this paper, we
analyze the dynamics of information transfer, or transference, across tasks
throughout training. Specifically, we develop a similarity measure that can
quantify transference among tasks and use this quantity to both better
understand the optimization dynamics of multi-task learning as well as improve
overall learning performance. In the latter case, we propose two methods to
leverage our transference metric. The first operates at a macro-level by
selecting which tasks should train together while the second functions at a
micro-level by determining how to combine task gradients at each training step.
We find these methods can lead to significant improvement over prior work on
three supervised multi-task learning benchmarks and one multi-task
reinforcement learning paradigm.

本文分析了多任务学习的信息传递动态，并开发了可量化任务间信息转移的相似度度量方法，从而使得我们在宏观和微观层面上提出了两种方法来优化多任务学习性能，这些方法在三个监督式多任务学习基准和一个多任务强化学习范式上都有显著改进。