Intelligent reflecting surface (IRS) is a promising technology to assist
downlink information transmissions from a multi-antenna access point (AP) to a
receiver. In this paper, we minimize the AP's transmit power by a joint
optimization of the AP's active beamforming and the IRS's passive beamforming.
Due to uncertain channel conditions, we formulate a robust power minimization
problem subject to the receiver's signal-to-noise ratio (SNR) requirement and
the IRS's power budget constraint. We propose a deep reinforcement learning
(DRL) approach that can adapt the beamforming strategies from past experiences.
To improve the learning performance, we derive a convex approximation as a
lower bound on the robust problem, which is integrated into the DRL framework
and thus promoting a novel optimization-driven deep deterministic policy
gradient (DDPG) approach. In particular, when the DDPG algorithm generates a
part of the action (e.g., passive beamforming), we can use the model-based
convex approximation to optimize the other part (e.g., active beamforming) of
the action more efficiently. Our simulation results demonstrate that the
optimization-driven DDPG algorithm can improve both the learning rate and
reward performance significantly compared to the conventional model-free DDPG
algorithm.

该文章提出一种利用智能反射面技术的利用无线电波在多天线接入点和接收器间进行信息传输的无线通信方案，并运用深度强化学习来优化天线束成形策略，从而最小化天线接入点的传输功率。

基于优化的深度强化学习在 IRS 辅助无线通信中的鲁棒波束成形

Optimization-driven Deep Reinforcement Learning for Robust Beamforming  in IRS-assisted Wireless Communications

Considering a Manhattan mobility model in vehicle-to-vehicle networks, this
work studies a power minimization problem subject to second-order statistical
constraints on latency and reliability, captured by a network-wide maximal data
queue length. We invoke results in extreme value theory to characterize
statistics of extreme events in terms of the maximal queue length.
Subsequently, leveraging Lyapunov stochastic optimization to deal with network
dynamics, we propose two queue-aware power allocation solutions. In contrast
with the baseline, our approaches achieve lower mean and variance of the
maximal queue length.

本文研究了在车辆之间网络中，基于曼哈顿机动性模型的功率最小化问题，同时考虑延迟和可靠性等二阶统计限制，最终提出了两种队列感知的功率分配方案，在降低队列最大长度的均值和方差方面取得了较优的结果。