In the past few years, AlphaZero's exceptional capability in mastering
intricate board games has garnered considerable interest. Initially designed
for the game of Go, this revolutionary algorithm merges deep learning
techniques with the Monte Carlo tree search (MCTS) to surpass earlier top-tier
methods. In our study, we broaden the use of AlphaZero to Gomoku, an age-old
tactical board game also referred to as "Five in a Row." Intriguingly, Gomoku
has innate challenges due to a bias towards the initial player, who has a
theoretical advantage. To add value, we strive for a balanced game-play. Our
tests demonstrate AlphaZero's versatility in adapting to games other than Go.
MCTS has become a predominant algorithm for decision processes in intricate
scenarios, especially board games. MCTS creates a search tree by examining
potential future actions and uses random sampling to predict possible results.
By leveraging the best of both worlds, the AlphaZero technique fuses deep
learning from Reinforcement Learning with the balancing act of MCTS,
establishing a fresh standard in game-playing AI. Its triumph is notably
evident in board games such as Go, chess, and shogi.

AlphaZero 算法通过将深度学习与蒙特卡洛树搜索相结合，在象棋、围棋和五子棋等棋盘游戏中取得了巨大的成功。

AlphaZero 五子棋

AlphaZero Gomoku

We consider the popular tree-based search strategy within the framework of
reinforcement learning, the Monte Carlo Tree Search (MCTS), in the context of
finite-horizon Markov decision process. We propose a dynamic sampling tree
policy that efficiently allocates limited computational budget to maximize the
probability of correct selection of the best action at the root node of the
tree. Experimental results on Tic-Tac-Toe and Gomoku show that the proposed
tree policy is more efficient than other competing methods.

本文探讨在有限时间马尔可夫决策过程的框架下，基于树形搜索策略的蒙特卡罗树搜索 (MCTS)。提出了一种动态抽样树策略，有效地分配有限的计算预算，以最大化选择最佳根节点动作的正确性概率。实验结果表明，所提出的树策略比其他竞争方法更有效。