We investigate the problem of learning an $\epsilon$-approximate solution for
the discrete-time Linear Quadratic Regulator (LQR) problem via a Stochastic
Variance-Reduced Policy Gradient (SVRPG) approach. Whilst policy gradient
methods have proven to converge linearly to the optimal solution of the
model-free LQR problem, the substantial requirement for two-point cost queries
in gradient estimations may be intractable, particularly in applications where
obtaining cost function evaluations at two distinct control input
configurations is exceptionally costly. To this end, we propose an
oracle-efficient approach. Our method combines both one-point and two-point
estimations in a dual-loop variance-reduced algorithm. It achieves an
approximate optimal solution with only
$O\left(\log\left(1/\epsilon\right)^{\beta}\right)$ two-point cost information
for $\beta \in (0,1)$.

通过随机方差缩减策略梯度方法，我们研究了离散时间线性二次调节器（LQR）问题的学习 ε- 近似解的问题。我们提出了一种适用于有高昂成本的成本函数评估的 Oracle - 有效方法，结合了一点和两点估计的方差缩减算法，在 β ∈ (0,1) 的情况下，仅需 O (log (1/ε)^β) 的两点成本信息即可获得近似最优解。