Planning problems are among the most important and well-studied problems in
artificial intelligence. They are most typically solved by tree search
algorithms that simulate ahead into the future, evaluate future states, and
back-up those evaluations to the root of a search tree. Among these algorithms,
Monte-Carlo tree search (MCTS) is one of the most general, powerful and widely
used. A typical implementation of MCTS uses cleverly designed rules, optimized
to the particular characteristics of the domain. These rules control where the
simulation traverses, what to evaluate in the states that are reached, and how
to back-up those evaluations. In this paper we instead learn where, what and
how to search. Our architecture, which we call an MCTSnet, incorporates
simulation-based search inside a neural network, by expanding, evaluating and
backing-up a vector embedding. The parameters of the network are trained
end-to-end using gradient-based optimisation. When applied to small searches in
the well known planning problem Sokoban, the learned search algorithm
significantly outperformed MCTS baselines.

该论文提出了一种名为 MCTSnet 的体系架构，其将基于模拟的搜索嵌入神经网络中，并通过向量嵌入扩展，评估和备份。该网络的参数进行端到端的训练优化，应用于小范围的搜索中，显著优于 MCTS 基线的性能。