The dominant line of work in domain adaptation has focused on learning invariant representations using domain-adversarial training. In this paper, we interpret this approach from a game theoretical perspective. Defining optimal solutions in domain-adversarial training as a local Nash equilibrium, we show that gradient descent in domain-adversarial training can violate the asymptotic convergence guarantees of the optimizer, oftentimes hindering the transfer performance. Our analysis leads us to replace gradient descent with high-order ODE solvers (i.e., Runge-Kutta), for which we derive asymptotic convergence guarantees. This family of optimizers is significantly more stable and allows more aggressive learning rates, leading to high performance gains when used as a drop-in replacement over standard optimizers. Our experiments show that in conjunction with state-of-the-art domain-adversarial methods, we achieve up to 3.5% improvement with less than of half training iterations. Our optimizers are easy to implement, free of additional parameters, and can be plugged into any domain-adversarial framework.

本文从博弈论的角度解释了域自适应训练中学习不变表示的支配性思路，并将梯度下降的优化器替换成高阶ODE求解器，为此得出渐近收敛保证。实验结果表明，与标准优化器相比，使用我们的优化器能够在半数训练迭代次数内，与最先进的域自适应方法相结合实现3.5％的性能提升。

面向领域的对抗性训练：博弈视角