Quantum computing (QC) in the current NISQ-era is still limited. To gain
early insights and advantages, hybrid applications are widely considered
mitigating those shortcomings. Hybrid quantum machine learning (QML) comprises
both the application of QC to improve machine learning (ML), and the
application of ML to improve QC architectures. This work considers the latter,
focusing on leveraging reinforcement learning (RL) to improve current QC
approaches. We therefore introduce various generic challenges arising from
quantum architecture search and quantum circuit optimization that RL algorithms
need to solve to provide benefits for more complex applications and
combinations of those. Building upon these challenges we propose a concrete
framework, formalized as a Markov decision process, to enable to learn policies
that are capable of controlling a universal set of quantum gates. Furthermore,
we provide benchmark results to assess shortcomings and strengths of current
state-of-the-art algorithms.

该研究论文介绍了量子计算在当前 NISQ 时代的限制，并提出了通过混合量子机器学习来改善量子计算架构的方法，重点是利用强化学习来优化当前的量子计算方法，并介绍了由量子架构搜索和量子电路优化引起的各种挑战，以及提出的用于学习控制一组万能量子门的具体框架，并提供基准结果以评估当前最先进算法的优点和短处。

量子计算中强化学习的挑战

Challenges for Reinforcement Learning in Quantum Computing

A central aspect for operating future quantum computers is quantum circuit
optimization, i.e., the search for efficient realizations of quantum algorithms
given the device capabilities. In recent years, powerful approaches have been
developed which focus on optimizing the high-level circuit structure. However,
these approaches do not consider and thus cannot optimize for the hardware
details of the quantum architecture, which is especially important for
near-term devices. To address this point, we present an approach to quantum
circuit optimization based on reinforcement learning. We demonstrate how an
agent, realized by a deep convolutional neural network, can autonomously learn
generic strategies to optimize arbitrary circuits on a specific architecture,
where the optimization target can be chosen freely by the user. We demonstrate
the feasibility of this approach by training agents on 12-qubit random
circuits, where we find on average a depth reduction by 27% and a gate count
reduction by 15%. We examine the extrapolation to larger circuits than used for
training, and envision how this approach can be utilized for near-term quantum
devices.

本研究使用基于强化学习的方法对量子电路进行优化，通过深度卷积神经网络，实现了对于特定架构的任意量子电路的自主学习及优化。对 12 比特电路进行实验得出优化效果，平均深度降低 27％，门数降低 15％，并探讨了方法在近期量子设备上的可行性。

深度强化学习的量子电路优化

Quantum circuit optimization with deep reinforcement learning

Variational quantum algorithms (VQAs) optimize the parameters $\vec{\theta}$
of a parametrized quantum circuit $V(\vec{\theta})$ to minimize a cost function
$C$. While VQAs may enable practical applications of noisy quantum computers,
they are nevertheless heuristic methods with unproven scaling. Here, we
rigorously prove two results, assuming $V(\vec{\theta})$ is an alternating
layered ansatz composed of blocks forming local 2-designs. Our first result
states that defining $C$ in terms of global observables leads to exponentially
vanishing gradients (i.e., barren plateaus) even when $V(\vec{\theta})$ is
shallow. Hence, several VQAs in the literature must revise their proposed
costs. On the other hand, our second result states that defining $C$ with local
observables leads to at worst a polynomially vanishing gradient, so long as the
depth of $V(\vec{\theta})$ is $\mathcal{O}(\log n)$. Our results establish a
connection between locality and trainability. We illustrate these ideas with
large-scale simulations, up to 100 qubits, of a quantum autoencoder
implementation.

本文研究了量子电路参数优化问题，通过对全局和局部可观测量定义了两种方法，证明了在一定条件下采用局部可观测量定义的方法能够获得更好的训练效果。