In Multi-task learning (MTL), a joint model is trained to simultaneously make
predictions for several tasks. Joint training reduces computation costs and
improves data efficiency; however, since the gradients of these different tasks
may conflict, training a joint model for MTL often yields lower performance
than its corresponding single-task counterparts. A common method for
alleviating this issue is to combine per-task gradients into a joint update
direction using a particular heuristic. In this paper, we propose viewing the
gradients combination step as a bargaining game, where tasks negotiate to reach
an agreement on a joint direction of parameter update. Under certain
assumptions, the bargaining problem has a unique solution, known as the Nash
Bargaining Solution, which we propose to use as a principled approach to
multi-task learning. We describe a new MTL optimization procedure, Nash-MTL,
and derive theoretical guarantees for its convergence. Empirically, we show
that Nash-MTL achieves state-of-the-art results on multiple MTL benchmarks in
various domains.

本文介绍了一种新的多任务学习优化过程 Nash-MTL，提出了一种将多个任务之间的梯度组合看作为协商议价游戏的方法，并将合理的 Nash 议价解作为多任务联合优化的原则方法。在实验中，我们显示 Nash-MTL 取得了各领域多个多任务学习基准测试的最佳结果。