Monte Carlo Tree Search (MCTS) is a powerful approach to designing
game-playing bots or solving sequential decision problems. The method relies on
intelligent tree search that balances exploration and exploitation. MCTS
performs random sampling in the form of simulations and stores statistics of
actions to make more educated choices in each subsequent iteration. The method
has become a state-of-the-art technique for combinatorial games, however, in
more complex games (e.g. those with high branching factor or real-time ones),
as well as in various practical domains (e.g. transportation, scheduling or
security) an efficient MCTS application often requires its problem-dependent
modification or integration with other techniques. Such domain-specific
modifications and hybrid approaches are the main focus of this survey. The last
major MCTS survey has been published in 2012. Contributions that appeared since
its release are of particular interest for this review.

本文调查了 Monte Carlo Tree Search (MCTS) 在领域特定修改和混合方法方面的应用，这种方法依赖于智能树搜索并平衡探索和利用。

蒙特卡洛树搜索：近期修改和应用综述

Monte Carlo Tree Search: A Review of Recent Modifications and  Applications

Deep reinforcement learning has achieved many recent successes, but our
understanding of its strengths and limitations is hampered by the lack of rich
environments in which we can fully characterize optimal behavior, and
correspondingly diagnose individual actions against such a characterization.
Here we consider a family of combinatorial games, arising from work of Erdos,
Selfridge, and Spencer, and we propose their use as environments for evaluating
and comparing different approaches to reinforcement learning. These games have
a number of appealing features: they are challenging for current learning
approaches, but they form (i) a low-dimensional, simply parametrized
environment where (ii) there is a linear closed form solution for optimal
behavior from any state, and (iii) the difficulty of the game can be tuned by
changing environment parameters in an interpretable way. We use these
Erdos-Selfridge-Spencer games not only to compare different algorithms, but
test for generalization, make comparisons to supervised learning, analyse
multiagent play, and even develop a self play algorithm. Code can be found at:
this https URL

本文提出利用由 Erdos，Selfridge 和 Spencer 为代表的组合博弈作为用于评估和比较强化学习的不同方法的环境，这些游戏具有低维度、简单参数化的特征，在任何状态下都有最优行为的线性闭合形式解决方案，可以通过改变环境参数来调整游戏难度。研究利用组合博弈比较不同的算法，测试泛化，与监督学习进行比较，分析多代理游戏，以及发展自我玩算法。

深度强化学习能否解决 Erdos-Selfridge-Spencer 游戏？

Can Deep Reinforcement Learning Solve Erdos-Selfridge-Spencer Games?

We study variations on combinatorial games in which, instead of alternating
moves, the players bid with discrete bidding chips for the right to determine
who moves next. We consider both symmetric and partisan games, and explore
differences between discrete bidding games and Richman games, which allow
real-valued bidding. Unlike Richman games, discrete bidding game variations of
many familiar games, such as chess, Connect Four, and even Tic-Tac-Toe, are
suitable for recreational play. We also present an analysis of Tic-Tac-Toe for
both discrete and real-valued bidding.

研究以离散出价替代交替走棋的组合游戏变化，分析这种游戏与 Richman 游戏的不同之处，并评估其在休闲游戏中的应用，包括黑白棋、四连棋以及井字游戏的竞价策略。