In this paper, we study the following nonlinear matrix decomposition (NMD)
problem: given a sparse nonnegative matrix $X$, find a low-rank matrix $\Theta$
such that $X \approx f(\Theta)$, where $f$ is an element-wise nonlinear
function. We focus on the case where $f(\cdot) = \max(0, \cdot)$, the rectified
unit (ReLU) non-linear activation. We refer to the corresponding problem as
ReLU-NMD. We first provide a brief overview of the existing approaches that
were developed to tackle ReLU-NMD. Then we introduce two new algorithms: (1)
aggressive accelerated NMD (A-NMD) which uses an adaptive Nesterov
extrapolation to accelerate an existing algorithm, and (2) three-block NMD
(3B-NMD) which parametrizes $\Theta = WH$ and leads to a significant reduction
in the computational cost. We also propose an effective initialization strategy
based on the nuclear norm as a proxy for the rank function. We illustrate the
effectiveness of the proposed algorithms (available on gitlab) on synthetic and
real-world data sets.

研究使用 ReLU 元素级非线性函数的非线性矩阵分解问题，提出了两种新算法（A-NMD 和 3B-NMD），它们通过自适应 Nesterov 外推和参数化 θ=WH 的方法来加速计算。同时，为了代替秩函数，提出了一种基于核范数的有效的初始化策略，并在合成和真实世界数据集上证明了所提出算法的有效性