We study the problem of parameter-free stochastic optimization, inquiring
whether, and under what conditions, do fully parameter-free methods exist:
these are methods that achieve convergence rates competitive with optimally
tuned methods, without requiring significant knowledge of the true problem
parameters. Existing parameter-free methods can only be considered
``partially'' parameter-free, as they require some non-trivial knowledge of the
true problem parameters, such as a bound on the stochastic gradient norms, a
bound on the distance to a minimizer, etc. In the non-convex setting, we
demonstrate that a simple hyperparameter search technique results in a fully
parameter-free method that outperforms more sophisticated state-of-the-art
algorithms. We also provide a similar result in the convex setting with access
to noisy function values under mild noise assumptions. Finally, assuming only
access to stochastic gradients, we establish a lower bound that renders fully
parameter-free stochastic convex optimization infeasible, and provide a method
which is (partially) parameter-free up to the limit indicated by our lower
bound.

该文章研究了无参数随机优化问题，探讨在什么条件下能够存在完全无参数的方法，并且通过简单的超参数搜索技术，证明了在凸和非凸环境下，可以实现优于当前先进算法的完全无参数方法。同时，对于仅有随机梯度的情况，该研究还提供了基于下限的 (部分) 无参数方法。