Mobile edge computing (MEC) emerges recently as a promising solution to
relieve resource-limited mobile devices from computation-intensive tasks, which
enables devices to offload workloads to nearby MEC servers and improve the
quality of computation experience. Nevertheless, by considering a MEC system
consisting of multiple mobile users with stochastic task arrivals and wireless
channels in this paper, the design of computation offloading policies is
challenging to minimize the long-term average computation cost in terms of
power consumption and buffering delay. A deep reinforcement learning (DRL)
based decentralized dynamic computation offloading strategy is investigated to
build a scalable MEC system with limited feedback. Specifically, a continuous
action space-based DRL approach named deep deterministic policy gradient (DDPG)
is adopted to learn efficient computation offloading policies independently at
each mobile user. Thus, powers of both local execution and task offloading can
be adaptively allocated by the learned policies from each user's local
observation of the MEC system. Numerical results are illustrated to demonstrate
that efficient policies can be learned at each user, and performance of the
proposed DDPG based decentralized strategy outperforms the conventional deep
Q-network (DQN) based discrete power control strategy and some other greedy
strategies with reduced computation cost. Besides, the power-delay tradeoff is
also analyzed for both the DDPG based and DQN based strategies.

本文探讨了一种分散式的深度强化学习方法来优化移动边缘计算系统中的任务卸载策略，并分析了功耗和延迟之间的权衡关系。结果表明，该方法比传统的深度 Q 网络方法以及其他贪心策略能够更有效地降低资源消耗成本。

多用户移动边缘计算的分散式计算卸载：一种深度强化学习方法

Decentralized Computation Offloading for Multi-User Mobile Edge  Computing: A Deep Reinforcement Learning Approach

While mobile edge computing (MEC) alleviates the computation and power
limitations of mobile devices, additional latency is incurred when offloading
tasks to remote MEC servers. In this work, the power-delay tradeoff in the
context of task offloading is studied in a multi-user MEC scenario. In contrast
with current system designs relying on average metrics (e.g., the average queue
length and average latency), a novel network design is proposed in which
latency and reliability constraints are taken into account. This is done by
imposing a probabilistic constraint on users' task queue lengths and invoking
results from extreme value theory to characterize the occurrence of
low-probability events in terms of queue length (or queuing delay) violation.
The problem is formulated as a computation and transmit power minimization
subject to latency and reliability constraints, and solved using tools from
Lyapunov stochastic optimization. Simulation results demonstrate the
effectiveness of the proposed approach, while examining the power-delay
tradeoff and required computational resources for various computation
intensities.

在多用户移动边缘计算场景下，考虑延迟和可靠性约束，提出了一种概率约束方法，使用了极值理论来解决低概率事件的问题，旨在最小化计算和传输功率，并使用 Lyapunov 随机优化工具解决。模拟结果表明该方法的有效性，同时研究了不同计算强度所需的功率延迟权衡和计算资源。