Clustered federated learning (FL) has been shown to produce promising results
by grouping clients into clusters. This is especially effective in scenarios
where separate groups of clients have significant differences in the
distributions of their local data. Existing clustered FL algorithms are
essentially trying to group together clients with similar distributions so that
clients in the same cluster can leverage each other's data to better perform
federated learning. However, prior clustered FL algorithms attempt to learn
these distribution similarities indirectly during training, which can be quite
time consuming as many rounds of federated learning may be required until the
formation of clusters is stabilized. In this paper, we propose a new approach
to federated learning that directly aims to efficiently identify distribution
similarities among clients by analyzing the principal angles between the client
data subspaces. Each client applies a truncated singular value decomposition
(SVD) step on its local data in a single-shot manner to derive a small set of
principal vectors, which provides a signature that succinctly captures the main
characteristics of the underlying distribution. This small set of principal
vectors is provided to the server so that the server can directly identify
distribution similarities among the clients to form clusters. This is achieved
by comparing the similarities of the principal angles between the client data
subspaces spanned by those principal vectors. The approach provides a simple,
yet effective clustered FL framework that addresses a broad range of data
heterogeneity issues beyond simpler forms of Non-IIDness like label skews. Our
clustered FL approach also enables convergence guarantees for non-convex
objectives. Our code is available at this https URL.

本文提出了一种新的聚类联邦学习方法，通过分析客户端数据子空间之间的主角度直接快速识别客户端之间的分布相似性，从而形成聚类，从而有效解决了数据异质性问题。

聚类联邦学习中基于客户端数据子空间主角之间的高效分布相似性识别

Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces

We study the non-convex matrix factorization approach to matrix completion
via Riemannian geometry. Based on an optimization formulation over a
Grassmannian manifold, we characterize the landscape based on the notion of
principal angles between subspaces. For the fully observed case, our results
show that there is a region in which the cost is geodesically convex, and
outside of which all critical points are strictly saddle. We empirically study
the partially observed case based on our findings.

本文研究了使用 Riemannian 几何的非凸矩阵分解方法来完成矩阵补全的问题，并且基于 Grassmann 流形提出了一种优化模型来表征优化问题的解空间，并探究了其几何形态。在完全观测的情况下，我们证明有一定区域使得代价函数的形式是地质凸的，在这个区域之外，所有的拐点都是严格的鞍点。最后，我们结合实验结果研究了部分观测的情况。

矩阵分解的黎曼视角

Riemannian Perspective on Matrix Factorization

We resolve a basic problem on subspace distances that often arises in
applications: How can the usual Grassmann distance between equidimensional
subspaces be extended to subspaces of different dimensions? We show that a
natural solution is given by the distance of a point to a Schubert variety
within the Grassmannian. This distance reduces to the Grassmann distance when
the subspaces are equidimensional and does not depend on any embedding into a
larger ambient space. Furthermore, it has a concrete expression involving
principal angles, and is efficiently computable in numerically stable ways. Our
results are largely independent of the Grassmann distance --- if desired, it
may be substituted by any other common distances between subspaces. Our
approach depends on a concrete algebraic geometric view of the Grassmannian
that parallels the differential geometric perspective that is well-established
in applied and computational mathematics.

研究如何将等维子空间之间的 Grassmann 距离扩展到不同维度的情况下，通过一个 Schubert variety 到 Grassmannian 空间中的点的距离计算出了一种自然解决方案，使用主角度可以方便的计算该距离并且结果具有数值稳定性，该方法不依赖于 Grassmann 距离且可以使用其他常见的子空间距离。此方法基于 Grassmannian 的一种具体的代数几何视角。

Schubert 变量与不同维度子空间之间的距离

Schubert varieties and distances between subspaces of different  dimensions

Principal angles between subspaces (PABS) (also called canonical angles)
serve as a classical tool in mathematics, statistics, and applications, e.g.,
data mining. Traditionally, PABS are introduced via their cosines. The cosines
and sines of PABS are commonly defined using the singular value decomposition.
We utilize the same idea for the tangents, i.e., explicitly construct matrices,
such that their singular values are equal to the tangents of PABS, using
several approaches: orthonormal and non-orthonormal bases for subspaces, as
well as projectors. Such a construction has applications, e.g., in analysis of
convergence of subspace iterations for eigenvalue problems.

本文介绍了一种新的针对特征值问题子空间迭代收敛分析的方法，通过构造一些矩阵，使它们的奇异值等于特征向量间的正切值，并探讨了正交和非正交基、投影等多种构造方式。