Leveraging on recent advancements on adaptive methods for convex minimization
problems, this paper provides a linesearch-free proximal gradient framework for
globalizing the convergence of popular stepsize choices such as
Barzilai-Borwein and one-dimensional Anderson acceleration. This framework can
cope with problems in which the gradient of the differentiable function is
merely locally H\"older continuous. Our analysis not only encompasses but also
refines existing results upon which it builds. The theory is corroborated by
numerical evidence that showcases the synergetic interplay between fast
stepsize selections and adaptive methods.

通过对于凸最小化问题的自适应方法的最新进展的利用，本文提供了一种无需线搜索的近端梯度下降框架，用于全局化收敛于流行的步长选择，如 Barzilai-Borwein 和一维 Anderson 加速。该框架可以处理梯度可微函数只具有局部 Holder 连续性的问题。我们的分析不仅包含但也改进了现有结果，并以数值证据来证明快速步长选择和自适应方法之间的协同作用。

保障自适应方法：巴基莱 - 波尔温法和其他步长选择的全局收敛

Safeguarding adaptive methods: global convergence of Barzilai-Borwein  and other stepsize choices

The variance reduction class of algorithms including the representative ones,
SVRG and SARAH, have well documented merits for empirical risk minimization
problems. However, they require grid search to tune parameters (step size and
the number of iterations per inner loop) for optimal performance. This work
introduces `almost tune-free' SVRG and SARAH schemes equipped with i)
Barzilai-Borwein (BB) step sizes; ii) averaging; and, iii) the inner loop
length adjusted to the BB step sizes. In particular, SVRG, SARAH, and their BB
variants are first reexamined through an `estimate sequence' lens to enable new
averaging methods that tighten their convergence rates theoretically, and
improve their performance empirically when the step size or the inner loop
length is chosen large. Then a simple yet effective means to adjust the number
of iterations per inner loop is developed to enhance the merits of the proposed
averaging schemes and BB step sizes. Numerical tests corroborate the proposed
methods.

本研究提出了一种使用 Barzilai-Borwein 步长、平均值和内循环长度来调整参数的近乎无需调参的 SVRG 和 SARAH 算法，通过 “估计序列” 方法重新审视了这些算法，并在内循环过程中进行了简单而有效的迭代次数调整，提高了方法的性能。