We derive lower bounds on the Bayes risk in decentralized estimation, where
the estimator does not have direct access to the random samples generated
conditionally on the random parameter of interest, but only to the data
received from local processors that observe the samples. The received data are
subject to communication constraints, due to quantization and the noise in the
communication channels from the processors to the estimator. We first derive
general lower bounds on the Bayes risk using information-theoretic quantities,
such as mutual information, information density, small ball probability, and
differential entropy. We then apply these lower bounds to the decentralized
case, using strong data processing inequalities to quantify the contraction of
information due to communication constraints. We treat the cases of a single
processor and of multiple processors, where the samples observed by different
processors may be conditionally dependent given the parameter, for
noninteractive and interactive communication protocols. Our results recover and
improve recent lower bounds on the Bayes risk and the minimax risk for certain
decentralized estimation problems, where previously only conditionally
independent sample sets and noiseless channels have been considered. Moreover,
our results provide a general way to quantify the degradation of estimation
performance caused by distributing resources to multiple processors, which is
only discussed for specific examples in existing works.

本文针对分布式估计中存在通信约束的情况，使用信息理论等量化方法来推导出下边界，进而量化资源分配给多个处理器造成的估计性能下降。

关于分散估计中贝叶斯风险的信息论下限

Information-Theoretic Lower Bounds on Bayes Risk in Decentralized  Estimation

Efficient and robust algorithms for decentralized estimation in networks are
essential to many distributed systems. Whereas distributed estimation of sample
mean statistics has been the subject of a good deal of attention, computation
of $U$-statistics, relying on more expensive averaging over pairs of
observations, is a less investigated area. Yet, such data functionals are
essential to describe global properties of a statistical population, with
important examples including Area Under the Curve, empirical variance, Gini
mean difference and within-cluster point scatter. This paper proposes new
synchronous and asynchronous randomized gossip algorithms which simultaneously
propagate data across the network and maintain local estimates of the
$U$-statistic of interest. We establish convergence rate bounds of $O(1/t)$ and
$O(\log t / t)$ for the synchronous and asynchronous cases respectively, where
$t$ is the number of iterations, with explicit data and network dependent
terms. Beyond favorable comparisons in terms of rate analysis, numerical
experiments provide empirical evidence the proposed algorithms surpasses the
previously introduced approach.

该论文提出了新的同步和异步随机流言算法，可以在网络中传播数据并同时维护感兴趣的 $U$- 统计量的本地估计，同时建立了收敛速率界限，并在数值实验中提供实际证据表明这些算法优于之前引入的方法。

将 Gossip 算法扩展到分布式 U-Statistics 估计

Extending Gossip Algorithms to Distributed Estimation of U-Statistics

In this paper, a distributed stochastic approximation algorithm is studied.
Applications of such algorithms include decentralized estimation, optimization,
control or computing. The algorithm consists in two steps: a local step, where
each node in a network updates a local estimate using a stochastic
approximation algorithm with decreasing step size, and a gossip step, where a
node computes a local weighted average between its estimates and those of its
neighbors. Convergence of the estimates toward a consensus is established under
weak assumptions. The approach relies on two main ingredients: the existence of
a Lyapunov function for the mean field in the agreement subspace, and a
contraction property of the random matrices of weights in the subspace
orthogonal to the agreement subspace. A second order analysis of the algorithm
is also performed under the form of a Central Limit Theorem. The
Polyak-averaged version of the algorithm is also considered.

本文研究了一种分布式随机逼近算法，可应用于去中心化的估算、优化、控制或计算。该算法包含本地步骤和交换步骤，通过微小的步长逼近算法和局部加权平均计算。通过研究约束子空间内平均场的 Lyapunov 函数和随机权重矩阵的收缩性质，建立估计向共识收敛的证明，并进行了二阶分析和多项式平均版本的研究。