Gradient descent (GD) is crucial for generalization in machine learning
models, as it induces implicit regularization, promoting compact
representations. In this work, we examine the role of GD in inducing implicit
regularization for tensor optimization, particularly within the context of the
lifted matrix sensing framework. This framework has been recently proposed to
address the non-convex matrix sensing problem by transforming spurious
solutions into strict saddles when optimizing over symmetric, rank-1 tensors.
We show that, with sufficiently small initialization scale, GD applied to this
lifted problem results in approximate rank-1 tensors and critical points with
escape directions. Our findings underscore the significance of the tensor
parametrization of matrix sensing, in combination with first-order methods, in
achieving global optimality in such problems.

通过梯度下降算法，我们探讨了诱导隐式正则化的作用，特别是在张量优化中的角色，尤其是在提升矩阵感知框架中。我们的发现强调了在与一阶方法相结合的矩阵感知中，张量参数化在实现全局最优性方面的重要性。