This research explores the trainability of Parameterized Quantum
circuit-based policies in Reinforcement Learning, an area that has recently
seen a surge in empirical exploration. While some studies suggest improved
sample complexity using quantum gradient estimation, the efficient trainability
of these policies remains an open question. Our findings reveal significant
challenges, including standard Barren Plateaus with exponentially small
gradients and gradient explosion. These phenomena depend on the type of
basis-state partitioning and mapping these partitions onto actions. For a
polynomial number of actions, a trainable window can be ensured with a
polynomial number of measurements if a contiguous-like partitioning of
basis-states is employed. These results are empirically validated in a
multi-armed bandit environment.

研究了参数化量子电路为基础的强化学习策略的可训练性，发现拥有指数小梯度和梯度爆炸的标准荒原问题，以及这些现象与基态分区和分区映射相关，采用连续型分区的基态可以确保多项式数量的训练窗口和测量次数，该研究在多臂赌博机环境中进行了实证验证。