Remote state estimation of large-scale distributed dynamic processes plays an
important role in Industry 4.0 applications. In this paper, by leveraging the
theoretical results of structural properties of optimal scheduling policies, we
develop a structure-enhanced deep reinforcement learning (DRL) framework for
optimal scheduling of a multi-sensor remote estimation system to achieve the
minimum overall estimation mean-square error (MSE). In particular, we propose a
structure-enhanced action selection method, which tends to select actions that
obey the policy structure. This explores the action space more effectively and
enhances the learning efficiency of DRL agents. Furthermore, we introduce a
structure-enhanced loss function to add penalty to actions that do not follow
the policy structure. The new loss function guides the DRL to converge to the
optimal policy structure quickly. Our numerical results show that the proposed
structure-enhanced DRL algorithms can save the training time by 50% and reduce
the remote estimation MSE by 10% to 25%, when compared to benchmark DRL
algorithms.

本文研发了一种结构增强的深度强化学习算法用于最小化远程估计均方误差，实现在多传感器远程估计系统的最优调度，该算法可以比基准算法节省 50％的训练时间并将远程估计均方误差降低 10％至 25％。