Multi-access Edge Computing (MEC) can be implemented together with Open Radio
Access Network (O-RAN) over commodity platforms to offer low-cost deployment
and bring the services closer to end-users. In this paper, a joint O-RAN/MEC
orchestration using a Bayesian deep reinforcement learning (RL)-based framework
is proposed that jointly controls the O-RAN functional splits, the allocated
resources and hosting locations of the O-RAN/MEC services across
geo-distributed platforms, and the routing for each O-RAN/MEC data flow. The
goal is to minimize the long-term overall network operation cost and maximize
the MEC performance criterion while adapting possibly time-varying O-RAN/MEC
demands and resource availability. This orchestration problem is formulated as
Markov decision process (MDP). However, the system consists of multiple BSs
that share the same resources and serve heterogeneous demands, where their
parameters have non-trivial relations. Consequently, finding the exact model of
the underlying system is impractical, and the formulated MDP renders in a large
state space with multi-dimensional discrete action. To address such modeling
and dimensionality issues, a novel model-free RL agent is proposed for our
solution framework. The agent is built from Double Deep Q-network (DDQN) that
tackles the large state space and is then incorporated with action branching,
an action decomposition method that effectively addresses the multi-dimensional
discrete action with linear increase complexity. Further, an efficient
exploration-exploitation strategy under a Bayesian framework using Thomson
sampling is proposed to improve the learning performance and expedite its
convergence. Trace-driven simulations are performed using an O-RAN-compliant
model. The results show that our approach is data-efficient (i.e., converges
faster) and increases the returned reward by 32\% than its non-Bayesian
version.

提出了一种基于贝叶斯深度强化学习框架的联合 O-RAN/MEC 编排，用于控制 O-RAN 功能分割、分配资源和主机位置、O-RAN/MEC 数据流的路由，以最小化长期网络操作成本并最大化 MEC 性能指标。