Chung's lemma is a classical tool for establishing asymptotic convergence
rates of (stochastic) optimization methods under strong convexity-type
assumptions and appropriate polynomial diminishing step sizes. In this work, we
develop a generalized version of Chung's lemma, which provides a simple
non-asymptotic convergence framework for a more general family of step size
rules. We demonstrate broad applicability of the proposed generalized Chung's
lemma by deriving tight non-asymptotic convergence rates for a large variety of
stochastic methods. In particular, we obtain partially new non-asymptotic
complexity results for stochastic optimization methods, such as stochastic
gradient descent and random reshuffling, under a general
$(\theta,\mu)$-Polyak-Lojasiewicz (PL) condition and for various step sizes
strategies, including polynomial, constant, exponential, and cosine step sizes
rules. Notably, as a by-product of our analysis, we observe that exponential
step sizes can adapt to the objective function's geometry, achieving the
optimal convergence rate without requiring exact knowledge of the underlying
landscape. Our results demonstrate that the developed variant of Chung's lemma
offers a versatile, systematic, and streamlined approach to establish
non-asymptotic convergence rates under general step size rules.

发展了一个泛化版本的 Chung 引理，用于建立非渐进收敛速度的收敛框架，适用于更一般的步长规则，并为各种随机方法提供了紧致的非渐进收敛速度。