Intelligent reflecting surface (IRS) is envisioned to be widely applied in
future wireless networks. In this paper, we investigate a multi-user
communication system assisted by cooperative IRS devices with the capability of
energy harvesting. Aiming to maximize the long-term average achievable system
rate, an optimization problem is formulated by jointly designing the transmit
beamforming at the base station (BS) and discrete phase shift beamforming at
the IRSs, with the constraints on transmit power, user data rate requirement
and IRS energy buffer size. Considering time-varying channels and stochastic
arrivals of energy harvested by the IRSs, we first formulate the problem as a
Markov decision process (MDP) and then develop a novel multi-agent Q-mix (MAQ)
framework with two layers to decouple the optimization parameters. The higher
layer is for optimizing phase shift resolutions, and the lower one is for phase
shift beamforming and power allocation. Since the phase shift optimization is
an integer programming problem with a large-scale action space, we improve MAQ
by incorporating the Wolpertinger method, namely, MAQ-WP algorithm to achieve a
sub-optimality with reduced dimensions of action space. In addition, as MAQ-WP
is still of high complexity to achieve good performance, we propose a policy
gradient-based MAQ algorithm, namely, MAQ-PG, by mapping the discrete phase
shift actions into a continuous space at the cost of a slight performance loss.
Simulation results demonstrate that the proposed MAQ-WP and MAQ-PG algorithms
can converge faster and achieve data rate improvements of 10.7% and 8.8% over
the conventional multi-agent DDPG, respectively.

本文提出了几种算法来最大化基站和用户之间通过能量采集协作 IRS 设备实现长期平均可实现系统速率，并考虑到时变信道和能量采集随机性，并改进了现有算法，以在减小行动空间的同时实现次优性能。模拟结果表明，所提出的算法可使数据传输速率提高 10.7％至 8.8％。

多智能体强化学习的协同智能反射面网络

Collaborative Intelligent Reflecting Surface Networks with Multi-Agent Reinforcement Learning

As a potential technology feature for 6G wireless networks, the idea of
sensing-communication integration requires the system not only to complete
reliable multi-user communication but also to achieve accurate environment
sensing. In this paper, we consider such a joint communication and sensing
(JCAS) scenario, in which multiple users use the sparse code multiple access
(SCMA) scheme to communicate with the wireless access point (AP). Part of the
user signals are scattered by the environment object and reflected by an
intelligent reflective surface (IRS) before they arrive at the AP. We exploit
the sparsity of both the structured user signals and the unstructured
environment and propose an iterative and incremental joint multi-user
communication and environment sensing scheme, in which the two processes, i.e.,
multi-user information detection and environment object detection, interweave
with each other thanks to their intrinsic mutual dependence. The proposed
algorithm is sliding-window based and also graph based, which can keep on
sensing the environment as long as there are illuminating user signals. The
trade-off relationship between the key system parameters is analyzed, and the
simulation result validates the convergence and effectiveness of the proposed
algorithm.

本文研究了在基于稀疏码多址接入的多用户通信系统中，利用环境物体和智能反射面迭代地进行多用户通信和环境感知的算法，并分析了关键系统参数之间的权衡关系和算法在模拟环境下的收敛性和有效性。

利用信号和环境稀疏性的联合多用户通信和感知

Joint Multi-User Communication and Sensing Exploiting Both Signal and  Environment Sparsity

The adoption of a Reconfigurable Intelligent Surface (RIS) for downlink
multi-user communication from a multi-antenna base station is investigated in
this paper. We develop energy-efficient designs for both the transmit power
allocation and the phase shifts of the surface reflecting elements, subject to
individual link budget guarantees for the mobile users. This leads to
non-convex design optimization problems for which to tackle we propose two
computationally affordable approaches, capitalizing on alternating
maximization, gradient descent search, and sequential fractional programming.
Specifically, one algorithm employs gradient descent for obtaining the RIS
phase coefficients, and fractional programming for optimal transmit power
allocation. Instead, the second algorithm employs sequential fractional
programming for the optimization of the RIS phase shifts. In addition, a
realistic power consumption model for RIS-based systems is presented, and the
performance of the proposed methods is analyzed in a realistic outdoor
environment. In particular, our results show that the proposed RIS-based
resource allocation methods are able to provide up to $300\%$ higher energy
efficiency, in comparison with the use of regular multi-antenna
amplify-and-forward relaying.

研究了采用可重构智能表面 (RIS) 用于来自多天线基站的下行多用户通信。我们针对移动用户的个体链路预算保证，开发了能源有效的传输功率分配和表面反射元素的相位移位的设计。我们提出了两种算法来解决这些非凸设计优化问题。此外，我们提出了一种逼真的基于 RIS 的系统功耗模型，并在现实的户外环境中分析了所提出方法的性能。