The first algorithm for the Linear Quadratic (LQ) control problem with an
unknown system model, featuring a regret of $\mathcal{O}(\sqrt{T})$, was
introduced by Abbasi-Yadkori and Szepesv\'ari (2011). Recognizing the
computational complexity of this algorithm, subsequent efforts (see Cohen et
al. (2019), Mania et al. (2019), Faradonbeh et al. (2020a), and Kargin et
al.(2022)) have been dedicated to proposing algorithms that are computationally
tractable while preserving this order of regret. Although successful, the
existing works in the literature lack a fully adaptive exploration-exploitation
trade-off adjustment and require a user-defined value, which can lead to
overall regret bound growth with some factors. In this work, noticing this gap,
we propose the first fully adaptive algorithm that controls the number of
policy updates (i.e., tunes the exploration-exploitation trade-off) and
optimizes the upper-bound of regret adaptively. Our proposed algorithm builds
on the SDP-based approach of Cohen et al. (2019) and relaxes its need for a
horizon-dependant warm-up phase by appropriately tuning the regularization
parameter and adding an adaptive input perturbation. We further show that
through careful exploration-exploitation trade-off adjustment there is no need
to commit to the widely-used notion of strong sequential stability, which is
restrictive and can introduce complexities in initialization.

提出了一种解决具有未知系统模型的线性二次（LQ）控制问题的算法，其遗憾为 O (√T)，并在此基础上提出了首个完全自适应的算法，同时控制策略更新次数和自适应地优化遗憾上限，避免了计算复杂性问题。