This paper considers two-player zero-sum finite-horizon Markov games with simultaneous moves. The study focuses on the challenging settings where the value function or the model is parameterized by general function classes. Provably efficient algorithms for both decoupled and {coordinated} settings are developed. In the {decoupled} setting where the agent controls a single player and plays against an arbitrary opponent, we propose a new model-free algorithm. The sample complexity is governed by the Minimax Eluder dimension -- a new dimension of the function class in Markov games. As a special case, this method improves the state-of-the-art algorithm by a $\sqrt{d}$ factor in the regret when the reward function and transition kernel are parameterized with $d$-dimensional linear features. In the {coordinated} setting where both players are controlled by the agent, we propose a model-based algorithm and a model-free algorithm. In the model-based algorithm, we prove that sample complexity can be bounded by a generalization of Witness rank to Markov games. The model-free algorithm enjoys a $\sqrt{K}$-regret upper bound where $K$ is the number of episodes. Our algorithms are based on new techniques of alternate optimism.

本文主要研究带有参数化的一般函数类的两人零和有限时间跨度马尔科夫博弈，在研究中提出了可行的算法，包括基于模型的算法和无模型算法，并且在状态-动作对数$d$线性特征的情况下取得了比现有算法更好的效果，同时提出了最小极小规模的模型维度等概念来解决抽样复杂度的问题，最终得出了在模型上算法抽样复杂度可以通过将见证人等级推广到马尔科夫博弈来边界化。

零和马尔可夫博弈中通用的函数逼近