This paper considers minimizing the age-of-information (AoI) and transmit
power consumption in a vehicular network, where a roadside unit (RSU) provides
timely updates about a set of physical processes to vehicles. We consider
non-orthogonal multi-modal information dissemination, which is based on
superposed message transmission from RSU and successive interference
cancellation (SIC) at vehicles. The formulated problem is a multi-objective
mixed-integer nonlinear programming problem; thus, a Pareto-optimal front is
very challenging to obtain. First, we leverage the weighted-sum approach to
decompose the multi-objective problem into a set of multiple single-objective
sub-problems corresponding to each predefined objective preference weight.
Then, we develop a hybrid deep Q-network (DQN)-deep deterministic policy
gradient (DDPG) model to solve each optimization sub-problem respective to
predefined objective-preference weight. The DQN optimizes the decoding order,
while the DDPG solves the continuous power allocation. The model needs to be
retrained for each sub-problem. We then present a two-stage
meta-multi-objective reinforcement learning solution to estimate the Pareto
front with a few fine-tuning update steps without retraining the model for each
sub-problem. Simulation results illustrate the efficacy of the proposed
solutions compared to the existing benchmarks and that the meta-multi-objective
reinforcement learning model estimates a high-quality Pareto frontier with
reduced training time.

本文研究了在车辆网络中如何通过最小化信息时代和发送功率消耗来提供及时更新，采用非正交多模式信息传播方法，利用混合深度 Q 网络和深度确定性策略梯度模型解决多目标优化问题，并提出了一个两阶段元多目标强化学习解决方案来估计 Pareto 前沿。

车联网中的非正交年龄最优信息传播：一个基于元多目标强化学习的方法

Non-orthogonal Age-Optimal Information Dissemination in Vehicular  Networks: A Meta Multi-Objective Reinforcement Learning Approach

This paper investigates the problem of minimizing the age-of-information
(AoI) and transmit power consumption in a vehicular network, where a roadside
unit (RSU) provides timely updates about a set of physical processes to
vehicles. Each vehicle is interested in maintaining the freshness of its
information status about one or more physical processes. A framework is
proposed to optimize the decisions to unicast, multicast, broadcast, or not
transmit updates to vehicles as well as power allocations to minimize the AoI
and the RSU's power consumption over a time horizon. The formulated problem is
a mixed-integer nonlinear programming problem (MINLP), thus a global optimal
solution is difficult to achieve. In this context, we first develop an ant
colony optimization (ACO) solution which provides near-optimal performance and
thus serves as an efficient benchmark. Then, for real-time implementation, we
develop a deep reinforcement learning (DRL) framework that captures the
vehicles' demands and channel conditions in the state space and assigns
processes to vehicles through dynamic unicast-multicast scheduling actions.
Complexity analysis of the proposed algorithms is presented. Simulation results
depict interesting trade-offs between AoI and power consumption as a function
of the network parameters.

该论文研究了如何通过使用深度强化学习等方法，优化决策，减少信息年龄和传输功率，实现在路边单元（RSU）向车辆提供物理过程的及时更新的车辆网络中，并通过仿真结果展示了 AoI 和功耗之间的有趣的权衡关系。