Optimization over the set of matrices that satisfy $X^\top B X = I_p$,
referred to as the generalized Stiefel manifold, appears in many applications
involving sampled covariance matrices such as canonical correlation analysis
(CCA), independent component analysis (ICA), and the generalized eigenvalue
problem (GEVP). Solving these problems is typically done by iterative methods,
such as Riemannian approaches, which require a computationally expensive
eigenvalue decomposition involving fully formed $B$. We propose a cheap
stochastic iterative method that solves the optimization problem while having
access only to a random estimate of the feasible set. Our method does not
enforce the constraint in every iteration exactly, but instead it produces
iterations that converge to a critical point on the generalized Stiefel
manifold defined in expectation. The method has lower per-iteration cost,
requires only matrix multiplications, and has the same convergence rates as its
Riemannian counterparts involving the full matrix $B$. Experiments demonstrate
its effectiveness in various machine learning applications involving
generalized orthogonality constraints, including CCA, ICA, and GEVP.

我们提出了一种便宜的随机迭代方法，解决了通过对可行解的随机估计来优化在广义斯蒂弗尔流形上的问题。该方法具有较低的每次迭代成本，只需要进行矩阵乘法，并且具有与完整矩阵 B 相关的 Riemannian 方法相同的收敛速度。实验证明了其在包括 CCA、ICA 和 GEVP 在内的各种涉及广义正交性约束的机器学习应用中的有效性。