Massive random access of devices in the emerging Open Radio Access Network
(O-RAN) brings great challenge to the access control and management. Exploiting
the bursting nature of the access requests, sparse active user detection (SAUD)
is an efficient enabler towards efficient access management, but the sparsity
might be deteriorated in case of uncoordinated massive access requests. To
dynamically preserve the sparsity of access requests, a reinforcement-learning
(RL)-assisted scheme of closed-loop access control utilizing the access class
barring technique is proposed, where the RL policy is determined through
continuous interaction between the RL agent, i.e., a next generation node base
(gNB), and the environment. The proposed scheme can be implemented by the
near-real-time RAN intelligent controller (near-RT RIC) in O-RAN, supporting
rapid switching between heterogeneous vertical applications, such as mMTC and
uRLLC services. Moreover, a data-driven scheme of deep-RL-assisted SAUD is
proposed to resolve highly complex environments with continuous and
high-dimensional state and action spaces, where a replay buffer is applied for
automatic large-scale data collection. An actor-critic framework is formulated
to incorporate the strategy-learning modules into the near-RT RIC. Simulation
results show that the proposed schemes can achieve superior performance in both
access efficiency and user detection accuracy over the benchmark scheme for
different heterogeneous services with massive access requests.

提出了基于强化学习的闭环访问控制方案，通过近实时的无线接入网络智能控制器与环境的持续交互来确定强化学习策略，以实现对 Open Radio Access Network 中大规模访问请求的高效控制和管理，同时还提出了数据驱动的深度强化学习辅助方案，可用于处理连续和高维状态、动作空间的高度复杂的环境。