In this paper we study the training dynamics for gradient flow on
over-parametrized tensor decomposition problems. Empirically, such training
process often first fits larger components and then discovers smaller
components, which is similar to a tensor deflation process that is commonly
used in tensor decomposition algorithms. We prove that for orthogonally
decomposable tensor, a slightly modified version of gradient flow would follow
a tensor deflation process and recover all the tensor components. Our proof
suggests that for orthogonal tensors, gradient flow dynamics works similarly as
greedy low-rank learning in the matrix setting, which is a first step towards
understanding the implicit regularization effect of over-parametrized models
for low-rank tensors.

本文研究了过度参数化张量分解问题上的梯度流训练动态。通过证明，在正交可分解的张量情况下，略微修改的梯度流会遵循张量缩减过程，并恢复所有张量分量。我们的证明表明，对于正交张量，梯度流动态的工作方式类似于矩阵情况下的贪心低秩学习，这是了解超参数模型对低秩张量的隐含正则化效应的第一步。