In this paper, we study the problem of dynamic channel allocation for URLLC
traffic in a multi-user multi-channel wireless network where urgent packets
have to be successfully transmitted in a timely manner. We formulate the
problem as a finite-horizon Markov Decision Process with a stochastic
constraint related to the QoS requirement, defined as the packet loss rate for
each user. We propose a novel weighted formulation that takes into account both
the total expected reward (number of successfully transmitted packets) and the
risk which we define as the QoS requirement violation. First, we use the value
iteration algorithm to find the optimal policy, which assumes a perfect
knowledge of the controller of all the parameters, namely the channel
statistics. We then propose a Q-learning algorithm where the controller learns
the optimal policy without having knowledge of neither the CSI nor the channel
statistics. We illustrate the performance of our algorithms with numerical
studies.

本研究针对多用户多信道无线网络中 URLLC 流动态信道分配问题，利用有穷时间马尔科夫决策过程框架，提出了一种新的基于风险加权的目标函数，利用值迭代算法和 Q-learning 算法分别求解了完美控制和不知晓 CSI 和信道参数时的最优策略，实验验证了算法的有效性。