This research will present a hybrid approach to accelerate convergence in a
second order optimization. An online finite difference approximation of the
diagonal Hessian matrix will be introduced, along with fuzzy inferencing of
several hyperparameters. Competitive results have been achieved

这篇研究介绍了一种混合方法，用于加快二阶优化中的收敛速度。介绍了对角黑塞矩阵的在线有限差分近似方法，以及对多个超参数的模糊推理。取得了竞争力的结果。

二阶优化中的模糊超参数更新

Fuzzy hyperparameters update in a second order optimization

We consider sketching algorithms which first compress data by multiplication
with a random sketch matrix, and then apply the sketch to quickly solve an
optimization problem, e.g., low-rank approximation and regression. In the
learning-based sketching paradigm proposed by~\cite{indyk2019learning}, the
sketch matrix is found by choosing a random sparse matrix, e.g., CountSketch,
and then the values of its non-zero entries are updated by running gradient
descent on a training data set. Despite the growing body of work on this
paradigm, a noticeable omission is that the locations of the non-zero entries
of previous algorithms were fixed, and only their values were learned. In this
work, we propose the first learning-based algorithms that also optimize the
locations of the non-zero entries. Our first proposed algorithm is based on a
greedy algorithm. However, one drawback of the greedy algorithm is its slower
training time. We fix this issue and propose approaches for learning a
sketching matrix for both low-rank approximation and Hessian approximation for
second order optimization. The latter is helpful for a range of constrained
optimization problems, such as LASSO and matrix estimation with a nuclear norm
constraint. Both approaches achieve good accuracy with a fast running time.
Moreover, our experiments suggest that our algorithm can still reduce the error
significantly even if we only have a very limited number of training matrices.

本文提出了优化稀疏矩阵的学习算法，通过优化矩阵中非零项的位置和值来实现低秩逼近、回归和二阶优化。