The use of reinforcement learning (RL) in scientific applications, such as
materials design and automated chemistry, is increasing. A major challenge,
however, lies in fact that measuring the state of the system is often costly
and time consuming in scientific applications, whereas policy learning with RL
requires a measurement after each time step. In this work, we make the
measurement costs explicit in the form of a costed reward and propose a
framework that enables off-the-shelf deep RL algorithms to learn a policy for
both selecting actions and determining whether or not to measure the current
state of the system at each time step. In this way, the agents learn to balance
the need for information with the cost of information. Our results show that
when trained under this regime, the Dueling DQN and PPO agents can learn
optimal action policies whilst making up to 50\% fewer state measurements, and
recurrent neural networks can produce a greater than 50\% reduction in
measurements. We postulate the these reduction can help to lower the barrier to
applying RL to real-world scientific applications.

本论文提出了一种框架，可使深度强化学习算法在科学应用方面实现选择行动和决定是否在每个时间步长测量系统当前状态的策略，以平衡获取信息和信息成本。结果表明，当在此模式下训练时，Dueling DQN 和 PPO 代理可以学习到最优行动策略，同时进行的状态测量可以降低 50％，而递归神经网络可以使测量降低 50％以上。