The high demand for data rate in the next generation of wireless
communication could be ensured by Non-Orthogonal Multiple Access (NOMA)
approach in the millimetre-wave (mmW) frequency band. Joint power allocation
and beamforming of mmW-NOMA systems is mandatory which could be met by
optimization approaches. To this end, we have exploited Deep Reinforcement
Learning (DRL) approach due to policy generation leading to an optimized
sum-rate of users. Actor-critic phenomena are utilized to measure the immediate
reward and provide the new action to maximize the overall Q-value of the
network. The immediate reward has been defined based on the summation of the
rate of two users regarding the minimum guaranteed rate for each user and the
sum of consumed power as the constraints. The simulation results represent the
superiority of the proposed approach rather than the Time-Division Multiple
Access (TDMA) and another NOMA optimized strategy in terms of sum-rate of
users.

该研究使用深度强化学习方法，对毫米波条件下的非正交多址接入进行了功率分配和波束成形的优化，以达到用户总速率最大化，并在模拟结果中表现出较其他方法更佳的效果。

深度强化学习的毫米波非正交多址下行系统联合功率分配和波束成形器

Joint Power Allocation and Beamformer for mmW-NOMA Downlink Systems by Deep Reinforcement Learning

High demand of data rate in the next generation of wireless communication
could be ensured by Non-Orthogonal Multiple Access (NOMA) approach in the
millimetre-wave (mmW) frequency band. Decreasing the interference on the other
users while maintaining the bit rate via joint power allocation and beamforming
is mandatory to guarantee the high demand of bit-rate. Furthermore, mmW
frequency bands dictates the hybrid structure for beamforming because of the
trade-off in implementation and performance, simultaneously. In this paper,
joint power allocation and hybrid beamforming of mmW-NOMA systems is brought up
via recent advances in machine learning and control theory approaches called
Deep Reinforcement Learning (DRL). Actor-critic phenomena is exploited to
measure the immediate reward and providing the new action to maximize the
overall Q-value of the network. Additionally, to improve the stability of the
approach, we have utilized Soft Actor-Critic (SAC) approach where overall
reward and action entropy is maximized, simultaneously. The immediate reward
has been defined based on the soft weighted summation of the rate of all the
users. The soft weighting is based on the achieved rate and allocated power of
each user. Furthermore, the channel responses between the users and base
station (BS) is defined as the state of environment, while action space is
involved of the digital and analog beamforming weights and allocated power to
each user. The simulation results represent the superiority of the proposed
approach rather than the Time-Division Multiple Access (TDMA) and Non-Line of
Sight (NLOS)-NOMA in terms of sum-rate of the users. It's outperformance is
caused by the joint optimization and independency of the proposed approach to
the channel responses.

本论文介绍了使用深度强化学习和控制理论方法 (DRL) 实现毫米波非正交多址接入 (NOMA) 系统的联合功率分配和混合波束成形，通过考虑用户之间的相互干扰并同时满足高速率需求，模拟结果表明该方法比时分复用多址 (TDMA) 和非线视距 (NLOS)-NOMA 方法在用户总速率方面效果更好，且该方法独立于信道响应。