Language model training in distributed settings is limited by the
communication cost of gradient exchanges. In this short note, we extend recent
work from Malladi et al. (2023), using shared randomness to perform distributed
fine-tuning with low bandwidth. The method is a natural decentralized extension
of memory-efficient Simultaneous Perturbation Stochastic Approximation (SPSA).
Each iteration, each machine seeds a Random Number Generator (RNG) to perform
local reproducible perturbations on model weights and calculate and exchange
scalar projected gradients, which are then used to update each model. By using
a (machine, sample) identifier as the random seed, each model can regenerate
one another's perturbations. As machines only exchange single-byte projected
gradients, this is highly communication efficient. There are also potential
privacy benefits, as projected gradients may be calculated on different
training data, and models never access the other's data. Our approach not only
drastically reduces communication bandwidth requirements but also accommodates
dynamic addition or removal of machines during the training process and retains
the memory-efficient and inference-only advantages of recent work. We perform
proof-of-concept experiments to demonstrate the potential usefulness of this
method, building off of rich literature on distributed optimization and
memory-efficient training.

本研究提出了一种基于共享随机性的分布式微调方法，通过使用单字节的梯度交换，减少了通信成本，并保留了最近研究中的内存效率和推理优势。

只需一个字节（每梯度）：关于使用共享随机性进行低带宽分散化语言模型微调的注记

Just One Byte (per gradient): A Note on Low-Bandwidth Decentralized  Language Model Finetuning Using Shared Randomness

We consider the fundamental problem of communicating an estimate of a real
number $x\in[0,1]$ using a single bit. A sender that knows $x$ chooses a value
$X\in\set{0,1}$ to transmit. In turn, a receiver estimates $x$ based on the
value of $X$. We consider both the biased and unbiased estimation problems and
aim to minimize the cost. For the biased case, the cost is the worst-case (over
the choice of $x$) expected squared error, which coincides with the variance if
the algorithm is required to be unbiased.
We first overview common biased and unbiased estimation approaches and prove
their optimality when no shared randomness is allowed. We then show how a small
amount of shared randomness, which can be as low as a single bit, reduces the
cost in both cases. Specifically, we derive lower bounds on the cost attainable
by any algorithm with unrestricted use of shared randomness and propose
near-optimal solutions that use a small number of shared random bits. Finally,
we discuss open problems and future directions.

本文提出了利用单个比特位通信来估算实数的方法，并通过共享随机比特位优化了成本。