The paper studies the problem of designing the Intelligent Reflecting Surface
(IRS) phase shifters for Multiple Input Single Output (MISO) communication
systems in spatiotemporally correlated channel environments, where the
destination can move within a confined area. The objective is to maximize the
expected sum of SNRs at the receiver over infinite time horizons. The problem
formulation gives rise to a Markov Decision Process (MDP). We propose a deep
actor-critic algorithm that accounts for channel correlations and destination
motion by constructing the state representation to include the current position
of the receiver and the phase shift values and receiver positions that
correspond to a window of previous time steps. The channel variability induces
high frequency components on the spectrum of the underlying value function. We
propose the preprocessing of the critic's input with a Fourier kernel which
enables stable value learning. Finally, we investigate the use of the
destination SNR as a component of the designed MDP state, which is common
practice in previous work. We provide empirical evidence that, when the
channels are spatiotemporally correlated, the inclusion of the SNR in the state
representation interacts with function approximation in ways that inhibit
convergence.

本文研究了设计智能反射面（IRS）相移器以适应时空相关信道环境中具有运动目标的多输入单输出（MISO）通信系统的问题。旨在最大化接收器无限时间内预期信噪比总和。提出了一种深度演员 - 评论家算法，将当前接收器位置和先前时间步窗口内对应于相移值和接收器位置的状态表示纳入考虑，以解决该问题。

空时相关环境中基于深度强化学习的智能表面反射相位设计

Deep Reinforcement Learning for IRS Phase Shift Design in Spatiotemporally Correlated Environments

Reconfigurable Intelligent Surfaces (RISs) have been recently considered as
an energy-efficient solution for future wireless networks due to their fast and
low-power configuration, which has increased potential in enabling massive
connectivity and low-latency communications. Accurate and low-overhead channel
estimation in RIS-based systems is one of the most critical challenges due to
the usually large number of RIS unit elements and their distinctive hardware
constraints. In this paper, we focus on the uplink of a RIS-empowered
multi-user Multiple Input Single Output (MISO) uplink communication systems and
propose a channel estimation framework based on the parallel factor
decomposition to unfold the resulting cascaded channel model. We present two
iterative estimation algorithms for the channels between the base station and
RIS, as well as the channels between RIS and users. One is based on alternating
least squares (ALS), while the other uses vector approximate message passing to
iteratively reconstruct two unknown channels from the estimated vectors. To
theoretically assess the performance of the ALS-based algorithm, we derived its
estimation Cram\'er-Rao Bound (CRB). We also discuss the downlink achievable
sum rate computation with estimated channels and different precoding schemes
for the base station. Our extensive simulation results show that our algorithms
outperform benchmark schemes and that the ALS technique achieves the CRB. It is
also demonstrated that the sum rate using the estimated channels always reach
that of perfect channels under various settings, thus, verifying the
effectiveness and robustness of the proposed estimation algorithms.

本研究提出了基于并行因式分解的 频道估计框架，通过交替最小二乘法 和向量近似传递消息的迭代估计算法， 实现了基站和 RIS 之间，以及 RIS 和用户之间 频道的估计，并探讨了基站不同预编码方案 完成下行可达速率计算。仿真结果表明， 所提出的算法优于基准方案，并证实了它的有效性 和鲁棒性。