We introduce AdaSub, a stochastic optimization algorithm that computes a
search direction based on second-order information in a low-dimensional
subspace that is defined adaptively based on available current and past
information. Compared to first-order methods, second-order methods exhibit
better convergence characteristics, but the need to compute the Hessian matrix
at each iteration results in excessive computational expenses, making them
impractical. To address this issue, our approach enables the management of
computational expenses and algorithm efficiency by enabling the selection of
the subspace dimension for the search. Our code is freely available on GitHub,
and our preliminary numerical results demonstrate that AdaSub surpasses popular
stochastic optimizers in terms of time and number of iterations required to
reach a given accuracy.

AdaSub 是一种基于二阶信息在低维子空间内计算搜索方向的随机优化算法。相比一阶方法，二阶方法具有更好的收敛特性，但每次迭代需要计算 Hessian 矩阵导致计算开销过大，因此不切实际。为解决这一问题，我们的方法通过选择搜索的子空间维度来管理计算开销和算法效率。我们的代码可以在 GitHub 上免费获取，并且我们的初步数值结果表明，AdaSub 在达到给定精度所需的时间和迭代次数方面优于流行的随机优化器。