Sequential decision-making under cost-sensitive tasks is prohibitively daunting, especially for the problem that has a significant impact on people's daily lives, such as malaria control, treatment recommendation. The main challenge faced by policymakers is to learn a policy from scratch by interacting with a complex environment in a few trials. This work introduces a practical, data-efficient policy learning method, named Variance-Bonus Monte Carlo Tree Search~(VB-MCTS), which can copy with very little data and facilitate learning from scratch in only a few trials. Specifically, the solution is a model-based reinforcement learning method. To avoid model bias, we apply Gaussian Process~(GP) regression to estimate the transitions explicitly. With the GP world model, we propose a variance-bonus reward to measure the uncertainty about the world. Adding the reward to the planning with MCTS can result in more efficient and effective exploration. Furthermore, the derived polynomial sample complexity indicates that VB-MCTS is sample efficient. Finally, outstanding performance on a competitive world-level RL competition and extensive experimental results verify its advantage over the state-of-the-art on the challenging malaria control task.

本文介绍了一种名为 Variance-Bonus Monte Carlo Tree Search (VB-MCTS) 的实用、数据效率高的策略学习方法，它是一种基于模型的强化学习方法，应用高斯过程回归估计转变，提出了一种方差奖励来衡量对世界的不确定性，进一步优化了计划，得到了更有效的探究，综合实验结果表明，在困难的疟疾控制任务上，VB-MCTS优于现有技术。

数据高效的疟疾控制强化学习