Self-play via online learning is one of the premier ways to solve large-scale
two-player zero-sum games, both in theory and practice. Particularly popular
algorithms include optimistic multiplicative weights update (OMWU) and
optimistic gradient-descent-ascent (OGDA). While both algorithms enjoy $O(1/T)$
ergodic convergence to Nash equilibrium in two-player zero-sum games, OMWU
offers several advantages including logarithmic dependence on the size of the
payoff matrix and $\widetilde{O}(1/T)$ convergence to coarse correlated
equilibria even in general-sum games. However, in terms of last-iterate
convergence in two-player zero-sum games, an increasingly popular topic in this
area, OGDA guarantees that the duality gap shrinks at a rate of
$O(1/\sqrt{T})$, while the best existing last-iterate convergence for OMWU
depends on some game-dependent constant that could be arbitrarily large. This
begs the question: is this potentially slow last-iterate convergence an
inherent disadvantage of OMWU, or is the current analysis too loose? Somewhat
surprisingly, we show that the former is true. More generally, we prove that a
broad class of algorithms that do not forget the past quickly all suffer the
same issue: for any arbitrarily small $\delta>0$, there exists a $2\times 2$
matrix game such that the algorithm admits a constant duality gap even after
$1/\delta$ rounds. This class of algorithms includes OMWU and other standard
optimistic follow-the-regularized-leader algorithms.

通过在线学习的自我对弈是解决大规模两人零和游戏的主要方法之一，尤其流行的算法包括乐观的乘积权重更新（OMWU）和乐观的梯度下降 - 梯度上升（OGDA），本文证明了 OMWU 存在潜在的较慢的最后迭代收敛问题。