TL;DR使用 Taylor 模式自动微分的近期泛化方法,我们提出了自动推导 majorizer 的优化器,这些通用的 Majorization-Minimization 优化器可应用于任意问题,并且从任何起始点收敛,无需超参数调整。
Abstract
majorization-minimization (MM) is a family of optimization methods that
iteratively reduce a loss by minimizing a locally-tight upper bound, called a
majorizer. Traditionally, majorizers were derived by hand, and