Recent studies incorporate Nesterov's accelerated gradient method for the
acceleration of gradient based training. The Nesterov's Accelerated
Quasi-Newton (NAQ) method has shown to drastically improve the convergence
speed compared to the conventional quasi-Newton method. This paper implements
NAQ for non-convex optimization on Tensorflow. Two modifications have been
proposed to the original NAQ algorithm to ensure global convergence and
eliminate linesearch. The performance of the proposed algorithm - mNAQ is
evaluated on standard non-convex function approximation benchmark problems and
microwave circuit modelling problems. The results show that the improved
algorithm converges better and faster compared to first order optimizers such
as AdaGrad, RMSProp, Adam, and the second order methods such as the
quasi-Newton method.

本文提出将 Nesterov 增量梯度法用于 Tensorflow 中的非凸优化，通过对原始算法进行两种修改，使其在标准非凸函数逼近基准问题和微波电路建模问题上表现出更好和更快的收敛性，该算法称为改进的 mNAQ 算法。