In Federated Learning (FL) paradigm, a parameter server (PS) concurrently
communicates with distributed participating clients for model collection,
update aggregation, and model distribution over multiple rounds, without
touching private data owned by individual clients. FL is appealing in
preserving data privacy; yet the communication between the PS and scattered
clients can be a severe bottleneck. Model compression algorithms, such as
quantization and sparsification, have been suggested but they generally assume
a fixed code length, which does not reflect the heterogeneity and variability
of model updates. In this paper, through both analysis and experiments, we show
strong evidences that variable-length is beneficial for compression in FL. We
accordingly present Fed-CVLC (Federated Learning Compression with
Variable-Length Codes), which fine-tunes the code length in response of the
dynamics of model updates. We develop optimal tuning strategy that minimizes
the loss function (equivalent to maximizing the model utility) subject to the
budget for communication. We further demonstrate that Fed-CVLC is indeed a
general compression design that bridges quantization and sparsification, with
greater flexibility. Extensive experiments have been conducted with public
datasets to demonstrate that Fed-CVLC remarkably outperforms state-of-the-art
baselines, improving model utility by 1.50%-5.44%, or shrinking communication
traffic by 16.67%-41.61%.

通过分析和实验证明可变长度对于 Federated Learning 中的压缩是有利的，因此提出了 Fed-CVLC 方法，用以根据模型更新动态来调整代码长度，最小化通信损耗并提高模型效用。通过实验证明 Fed-CVLC 优于现有基准方法，可以提升模型效用 1.50% 至 5.44%，或减少通信流量 16.67% 至 41.61%。

Fed-CVLC：变长编码压缩联邦学习通信

Fed-CVLC: Compressing Federated Learning Communications with  Variable-Length Codes

Given a finite alphabet $A$ and a binary relation $\tau\subseteq A^*\times
A^*$, a set $X$ is $\tau$-{\it independent} if $ \tau(X)\cap X=\emptyset$.
Given a quasi-metric $d$ over $A^*$ (in the meaning of \cite{W31}) and $k\ge
1$, we associate the relation $\tau_{d,k}$ defined by $(x,y)\in\tau_{d,k}$ if,
and only if, $d(x,y)\le k$ \cite{CP02}.In the spirit of \cite{JK97,N21}, the
error detection-correction capability of variable-length codes can be expressed
in term of conditions over $\tau_{d,k}$. With respect to the prefix metric, the
factor one, and every quasi-metric associated to (anti-)automorphisms of the
free monoid, we examine whether those conditions are decidable for a given
regular code.

研究了与自动机同构相关的几种赋范情况下，对于给定的正则码，能否判断基于 $\tau_{d,k}$ 的条件的变长编码的错误检测和纠正能力的问题。

变长编码与误差检测领域的交叉

When Variable-Length Codes Meet the Field of Error Detection

Constrained sequence (CS) codes, including fixed-length CS codes and
variable-length CS codes, have been widely used in modern wireless
communication and data storage systems. Sequences encoded with constrained
sequence codes satisfy constraints imposed by the physical channel to enable
efficient and reliable transmission of coded symbols. In this paper, we propose
using deep learning approaches to decode fixed-length and variable-length CS
codes. Traditional encoding and decoding of fixed-length CS codes rely on
look-up tables (LUTs), which is prone to errors that occur during transmission.
We introduce fixed-length constrained sequence decoding based on multiple layer
perception (MLP) networks and convolutional neural networks (CNNs), and
demonstrate that we are able to achieve low bit error rates that are close to
maximum a posteriori probability (MAP) decoding as well as improve the system
throughput. Further, implementation of capacity-achieving fixed-length codes,
where the complexity is prohibitively high with LUT decoding, becomes practical
with deep learning-based decoding. We then consider CNN-aided decoding of
variable-length CS codes. Different from conventional decoding where the
received sequence is processed bit-by-bit, we propose using CNNs to perform
one-shot batch-processing of variable-length CS codes such that an entire batch
is decoded at once, which improves the system throughput. Moreover, since the
CNNs can exploit global information with batch-processing instead of only
making use of local information as in conventional bit-by-bit processing, the
error rates can be reduced. We present simulation results that show excellent
performance with both fixed-length and variable-length CS codes that are used
in the frontiers of wireless communication systems.

本文提出使用深度学习方法解码固定长度和可变长度的约束序列码，其中基于多层感知机（MLP）网络和卷积神经网络（CNN）的解码固定长度的约束序列码和使用 CNN 对可变长度的约束序列码进行批处理解码，可以提高系统吞吐量和降低误比特率。