In this work, we propose a federated dynamical low-rank training (FeDLRT)
scheme to reduce client compute and communication costs - two significant
performance bottlenecks in horizontal federated learning. Our method builds
upon dynamical low-rank splitting schemes for manifold-constrained optimization
to create a global low-rank basis of network weights, which enables client
training on a small coefficient matrix. A consistent global low-rank basis
allows us to incorporate a variance correction scheme and prove global loss
descent and convergence to a stationary point. Dynamic augmentation and
truncation of the low-rank bases automatically optimizes computing and
communication resource utilization. We demonstrate the efficiency of FeDLRT in
an array of computer vision benchmarks and show a reduction of client compute
and communication costs by up to an order of magnitude with minimal impacts on
global accuracy.

我们提出了一种联邦动态低秩训练 (FeDLRT) 方案，用于减少水平联邦学习中的客户端计算和通信成本，并在计算机视觉基准测试中证明了 FeDLRT 的效率。