There are only a few learning algorithms applicable to stochastic dynamic
teams and games which generalize Markov decision processes to decentralized
stochastic control problems involving possibly self-interested decision makers.
Learning in games is generally difficult because of the non-stationary
environment in which each decision maker aims to learn its optimal decisions
with minimal information in the presence of the other decision makers who are
also learning. In stochastic dynamic games, learning is more challenging
because, while learning, the decision makers alter the state of the system and
hence the future cost. In this paper, we present decentralized Q-learning
algorithms for stochastic games, and study their convergence for the weakly
acyclic case which includes team problems as an important special case. The
algorithm is decentralized in that each decision maker has access to only its
local information, the state information, and the local cost realizations;
furthermore, it is completely oblivious to the presence of other decision
makers. We show that these algorithms converge to equilibrium policies almost
surely in large classes of stochastic games.

本文介绍了用于随机动态团队和游戏的分散 Q 学习算法，研究了其在包括团队问题在内的弱无环情况下的收敛性。