This paper studies semantic-aware communication for remote estimation of
multiple Markov sources over a lossy and rate-constrained channel. Unlike most
existing studies that treat all source states equally, we exploit the semantics
of information and consider that the remote actuator has different tolerances
for the estimation errors of different states. We aim to find an optimal
scheduling policy that minimizes the long-term state-dependent costs of
estimation errors under a transmission frequency constraint. We theoretically
show the structure of the optimal policy by leveraging the average-cost
Constrained Markov Decision Process (CMDP) theory and the Lagrangian dynamic
programming. By exploiting the optimal structural results, we develop a novel
policy search algorithm, termed intersection search plus relative value
iteration (Insec-RVI), that can find the optimal policy using only a few
iterations. To avoid the ``curse of dimensionality'' of MDPs, we propose an
online low-complexity drift-plus-penalty (DPP) scheduling algorithm based on
the Lyapunov optimization theorem. We also design an efficient average-cost
Q-learning algorithm to estimate the optimal policy without knowing a priori
the channel and source statistics. Numerical results show that continuous
transmission is inefficient, and remarkably, our semantic-aware policies can
attain the optimum by strategically utilizing fewer transmissions by exploiting
the timing of the important information.

该研究利用平均成本约束马尔可夫决策过程理论和 Lagrangian 动态规划，探索了在有损且受速率限制的通道上进行多个马尔可夫源的远程估计的语义感知通信；通过使用结构优化结果和开发新的策略搜索算法，可以最小化长期状态依赖成本和估计误差；为了避免马尔可夫决策过程的维度问题，提出了基于 Lyapunov 优化定理的在线低复杂度 Drift-plus-Penalty 调度算法。