The development of 6G/B5G wireless networks, which have requirements that go
beyond current 5G networks, is gaining interest from academic and industrial.
However, to increase 6G/B5G network quality, conventional cellular networks
that rely on terrestrial base stations are constrained geographically and
economically. Meanwhile, NOMA allows multiple users to share the same
resources, which improves the spectral efficiency of the system and has the
advantage of supporting a larger number of users. Additionally, by
intelligently manipulating the phase and amplitude of both the reflected and
transmitted signals, STAR-RISs can achieve improved coverage, increased
spectral efficiency, and enhanced communication reliability. However, STAR-RISs
must simultaneously optimize the Amplitude and Phase-shift corresponding to
reflection and transmission, which makes the existing terrestiral networks more
complicated and is considered a major challenging issue. Motivated by the
above, we study the joint user pairing for NOMA and beamforming design of
Multi-STAR-RISs in an indoor environment. Then, we formulate the optimization
problem with the objective of maximizing the total throughput of MUs by jointly
optimizing the decoding order, user pairing, active beamforming, and passive
beamforming. However, the formulated problem is a MINLP. To tackle this
challenge, we first introduce the decoding order for NOMA networks. Next, we
decompose the original problem into two subproblems namely: 1) MU pairing and
2) Beamforming optimization under the optimal decoding order. For the first
subproblem, we employ correlation-based K-means clustering to solve the user
pairing problem. Then, to jointly deal with beamforming vector optimizations,
we propose MAPPO, which can make quick decisions in the given environment owing
to its low complexity.

在室内环境中，研究了 NOMA 和 Multi-STAR-RISs 的联合用户配对和波束赋形设计，通过解决用户配对问题和波束赋形向量优化问题来最大化 MUs 的总吞吐量。

基于多智能体强化学习的室内多星辅助 NOMA 系统的联合用户分配和波束成形设计

Joint User Pairing and Beamforming Design of Multi-STAR-RISs-Aided NOMA  in the Indoor Environment via Multi-Agent Reinforcement Learning

Simultaneously transmitting and reflecting reconfigurable intelligent
surfaces (STAR-RISs) is a promising passive device that contributes to a
full-space coverage via transmitting and reflecting the incident signal
simultaneously. As a new paradigm in wireless communications, how to analyze
the coverage and capacity performance of STAR-RISs becomes essential but
challenging. To solve the coverage and capacity optimization (CCO) problem in
STAR-RIS assisted networks, a multi-objective proximal policy optimization
(MO-PPO) algorithm is proposed to handle long-term benefits than conventional
optimization algorithms. To strike a balance between each objective, the MO-PPO
algorithm provides a set of optimal solutions to form a Pareto front (PF),
where any solution on the PF is regarded as an optimal result. Moreover, in
order to improve the performance of the MO-PPO algorithm, two update
strategies, i.e., action-value-based update strategy (AVUS) and loss
function-based update strategy (LFUS), are investigated. For the AVUS, the
improved point is to integrate the action values of both coverage and capacity
and then update the loss function. For the LFUS, the improved point is only to
assign dynamic weights for both loss functions of coverage and capacity, while
the weights are calculated by a min-norm solver at every update. The numerical
results demonstrated that the investigated update strategies outperform the
fixed weights MO optimization algorithms in different cases, which includes a
different number of sample grids, the number of STAR-RISs, the number of
elements in the STAR-RISs, and the size of STAR-RISs. Additionally, the
STAR-RIS assisted networks achieve better performance than conventional
wireless networks without STAR-RISs. Moreover, with the same bandwidth,
millimeter wave is able to provide higher capacity than sub-6 GHz, but at a
cost of smaller coverage.

通过多目标策略优化算法提出一种优化方法，以实现 STAR-RIS 辅助网络中的覆盖率和容量性能的优化。数值结果表明，所研究的算法优于传统的 MO 优化算法在不同情况下，包括样本网格的数量，STAR-RIS 的数量，STAR-RIS 的元素数量和 STAR-RIS 的大小。而且，在相同的带宽下，毫米波能够提供比次 6 GHz 更高的容量，但代价是更小的覆盖范围。