Regularization for matrix factorization (MF) and approximation problems has been carried out in many different ways. Due to its popularity in deep learning, dropout has been applied also for this class of problems. Despite its solid empirical performance, the theoretical properties of dropout as a regularizer remain quite elusive for this class of problems. In this paper, we present a theoretical analysis of dropout for MF, where Bernoulli random variables are used to drop columns of the factors. We demonstrate the equivalence between dropout and a fully deterministic model for MF in which the factors are regularized by the sum of the product of squared Euclidean norms of the columns. Additionally, we inspect the case of a variable sized factorization and we prove that dropout achieves the global minimum of a convex approximation problem with (squared) nuclear norm regularization. As a result, we conclude that dropout can be used as a low-rank regularizer with data dependent singular-value thresholding.

本文理论分析了使用 dropout 作为低秩正则化器以解决矩阵分解与逼近问题的表现，证明了基于伯努利随机变量的 dropout 等价于 MF 的完全确定性模型，其中因素用列的平方欧几里得范数积之和进行正则化，同时在变尺寸的矩阵分解情况下，使用 dropout 可以实现带（平方）核范数正则化的凸逼近问题的全局最小值。

Dropout作为矩阵分解的低秩正则化器