Quantum Computing aims to streamline machine learning, making it more
effective with fewer trainable parameters. This reduction of parameters can
speed up the learning process and reduce the use of computational resources.
However, in the current phase of quantum computing development, known as the
noisy intermediate-scale quantum era (NISQ), learning is difficult due to a
limited number of qubits and widespread quantum noise. To overcome these
challenges, researchers are focusing on variational quantum circuits (VQCs).
VQCs are hybrid algorithms that merge a quantum circuit, which can be adjusted
through parameters, with traditional classical optimization techniques. These
circuits require only few qubits for effective learning. Recent studies have
presented new ways of applying VQCs to reinforcement learning, showing
promising results that warrant further exploration. This study investigates the
effects of various techniques -- data re-uploading, input scaling, output
scaling -- and introduces exponential learning rate decay in the quantum
proximal policy optimization algorithm's actor-VQC. We assess these methods in
the popular Frozen Lake and Cart Pole environments. Our focus is on their
ability to reduce the number of parameters in the VQC without losing
effectiveness. Our findings indicate that data re-uploading and an exponential
learning rate decay significantly enhance hyperparameter stability and overall
performance. While input scaling does not improve parameter efficiency, output
scaling effectively manages greediness, leading to increased learning speed and
robustness.

量子计算通过减少可训练参数来提高机器学习效果，并且通过使用变分量子电路 (VQCs) 融合经典优化技术，研究人员致力于在噪声中等规模量子时代 (NISQ) 中，应用 VQCs 到强化学习中以减少参数并提高超参数稳定性及整体性能。

强化学习中变分量子电路的优化技术研究

A Study on Optimization Techniques for Variational Quantum Circuits in  Reinforcement Learning

The moving target segmentation (MTS) aims to segment out moving targets in
the video, however, the classical algorithm faces the huge challenge of
real-time processing in the current video era. Some scholars have successfully
demonstrated the quantum advantages in some video processing tasks, but not
concerning moving target segmentation. In this paper, a quantum moving target
segmentation algorithm for grayscale video is proposed, which can use quantum
mechanism to simultaneously calculate the difference of all pixels in all
adjacent frames and then quickly segment out the moving target. In addition, a
feasible quantum comparator is designed to distinguish the grayscale values
with the threshold. Then several quantum circuit units, including three-frame
difference, binarization and AND operation, are designed in detail, and then
are combined together to construct the complete quantum circuits for segmenting
the moving target. For a quantum video with $2^m$ frames (every frame is a
$2^n\times 2^n$ image with $q$ grayscale levels), the complexity of our
algorithm can be reduced to O$(n^2 + q)$. Compared with the classic
counterpart, it is an exponential speedup, while its complexity is also
superior to the existing quantum algorithms. Finally, the experiment is
conducted on IBM Q to show the feasibility of our algorithm in the noisy
intermediate-scale quantum (NISQ) era.

提出了一种针对灰度视频的量子移动目标分割算法，通过使用量子机制同时计算所有相邻帧中所有像素的差异，快速分割出移动目标。同时设计了可行的量子比较器来区分灰度值与阈值，在细节上设计了包括三帧差分、二值化和与操作在内的多个量子电路单元，并将它们组合在一起构建完成的量子电路，用于分割移动目标。对于一个包含 $2^m$ 帧的量子视频（每帧是一个包含 $q$ 个灰度级的 $2^n	imes 2^n$ 图像），我们的算法的复杂度可以降低到 O$(n^2 + q)$。与经典算法相比，这是指数级的加速，同时其复杂性也优于现有的量子算法。最后，在 IBM Q 上进行的实验证明了我们的算法在噪声中等规模量子（NISQ）时代的可行性。