Regret-based algorithms are highly efficient at finding approximate Nash
equilibria in sequential games such as poker games. However, most regret-based
algorithms, including counterfactual regret minimization (CFR) and its
variants, rely on iterate averaging to achieve convergence. Inspired by recent
advances on last-iterate convergence of optimistic algorithms in zero-sum
normal-form games, we study this phenomenon in sequential games, and provide a
comprehensive study of last-iterate convergence for zero-sum extensive-form
games with perfect recall (EFGs), using various optimistic regret-minimization
algorithms over treeplexes. This includes algorithms using the vanilla entropy
or squared Euclidean norm regularizers, as well as their dilated versions which
admit more efficient implementation. In contrast to CFR, we show that all of
these algorithms enjoy last-iterate convergence, with some of them even
converging exponentially fast. We also provide experiments to further support
our theoretical results.

本文研究了基于遗憾的算法在连续游戏中寻找近似的纳什均衡，针对反事实遗憾最小化（CFR）算法存在的表示收敛的缺陷，提出了一些基于树形复合结构的乐观遗憾最小化算法，并给出了实验证明其在求解连续游戏时的有效性。

广义博弈中的最后迭代收敛

Last-iterate Convergence in Extensive-Form Games

We study the performance of optimistic regret-minimization algorithms for
both minimizing regret in, and computing Nash equilibria of, zero-sum
extensive-form games. In order to apply these algorithms to extensive-form
games, a distance-generating function is needed. We study the use of the
dilated entropy and dilated Euclidean distance functions. For the dilated
Euclidean distance function we prove the first explicit bounds on the
strong-convexity parameter for general treeplexes. Furthermore, we show that
the use of dilated distance-generating functions enable us to decompose the
mirror descent algorithm, and its optimistic variant, into local mirror descent
algorithms at each information set. This decomposition mirrors the structure of
the counterfactual regret minimization framework, and enables important
techniques in practice, such as distributed updates and pruning of cold parts
of the game tree. Our algorithms provably converge at a rate of $T^{-1}$, which
is superior to prior counterfactual regret minimization algorithms. We
experimentally compare to the popular algorithm CFR+, which has a theoretical
convergence rate of $T^{-0.5}$ in theory, but is known to often converge at a
rate of $T^{-1}$, or better, in practice. We give an example matrix game where
CFR+ experimentally converges at a relatively slow rate of $T^{-0.74}$, whereas
our optimistic methods converge faster than $T^{-1}$. We go on to show that our
fast rate also holds in the Kuhn poker game, which is an extensive-form game.
For games with deeper game trees however, we find that CFR+ is still faster.
Finally we show that when the goal is minimizing regret, rather than computing
a Nash equilibrium, our optimistic methods can outperform CFR+, even in deep
game trees.

研究了在广泛形式博弈中最小化后悔和计算纳什均衡的乐观后悔最小化算法的性能，研究了扩展形式游戏距离生成函数的使用，证明了扩展欧几里德距离函数具有广义树片段的强凸性参数的第一个显式边界，提出了一种乐观算法可以优化计算效率，这在最小化后悔而不是计算纳什均衡时表现出很好的结果。