This paper proposes an effective and novel multiagent deep reinforcement
learning (MADRL)-based method for solving the joint virtual network function
(VNF) placement and routing (P&R), where multiple service requests with
differentiated demands are delivered at the same time. The differentiated
demands of the service requests are reflected by their delay- and
cost-sensitive factors. We first construct a VNF P&R problem to jointly
minimize a weighted sum of service delay and resource consumption cost, which
is NP-complete. Then, the joint VNF P&R problem is decoupled into two iterative
subtasks: placement subtask and routing subtask. Each subtask consists of
multiple concurrent parallel sequential decision processes. By invoking the
deep deterministic policy gradient method and multi-agent technique, an
MADRL-P&R framework is designed to perform the two subtasks. The new joint
reward and internal rewards mechanism is proposed to match the goals and
constraints of the placement and routing subtasks. We also propose the
parameter migration-based model-retraining method to deal with changing network
topologies. Corroborated by experiments, the proposed MADRL-P&R framework is
superior to its alternatives in terms of service cost and delay, and offers
higher flexibility for personalized service demands. The parameter
migration-based model-retraining method can efficiently accelerate convergence
under moderate network topology changes.

本文提出了一种基于多智能体深度强化学习的有效而新颖的方法，用于解决联合虚拟网络功能（VNF）部署和路由问题，该问题涉及多个不同需求的服务请求同时交付，并通过其延迟和成本敏感因素反映服务请求的不同需求，提出了基于参数移植的模型重新训练方法来处理网络拓扑发生更改的情况。