This paper presents a novel hierarchical federated learning algorithm within
multiple sets that incorporates quantization for communication-efficiency and
demonstrates resilience to statistical heterogeneity. Unlike conventional
hierarchical federated learning algorithms, our approach combines gradient
aggregation in intra-set iterations with model aggregation in inter-set
iterations. We offer a comprehensive analytical framework to evaluate its
optimality gap and convergence rate, comparing these aspects with those of
conventional algorithms. Additionally, we develop a problem formulation to
derive optimal system parameters in a closed-form solution. Our findings reveal
that our algorithm consistently achieves high learning accuracy over a range of
parameters and significantly outperforms other hierarchical algorithms,
particularly in scenarios with heterogeneous data distributions.

本文提出了一种新颖的分层联邦学习算法，其在多个集合中结合了量化进行通信效率，并且展示了对统计异质性的鲁棒性。与传统的分层联邦学习算法不同，我们的方法将集合内迭代中的梯度聚合与集合间迭代中的模型聚合相结合。我们提供了一个全面的分析框架来评估其最优性差距和收敛速度，并将这些方面与传统算法进行比较。此外，我们还提出了一个问题形式化解决方案以推导出最优的系统参数。我们的研究结果表明，我们的算法在一系列参数下始终能够实现高学习准确率，并且在具有异构数据分布的场景中明显优于其他分层算法。

量化分层联邦学习：统计异质性的鲁棒方法

Quantized Hierarchical Federated Learning: A Robust Approach to  Statistical Heterogeneity

Spiking neural networks (SNNs) are bio-plausible computing models with high
energy efficiency. The temporal dynamics of neurons and synapses enable them to
detect temporal patterns and generate sequences. While Backpropagation Through
Time (BPTT) is traditionally used to train SNNs, it is not suitable for online
learning of embedded applications due to its high computation and memory cost
as well as extended latency. Previous works have proposed online learning
algorithms, but they often utilize highly simplified spiking neuron models
without synaptic dynamics and reset feedback, resulting in subpar performance.
In this work, we present Spatiotemporal Online Learning for Synaptic Adaptation
(SOLSA), specifically designed for online learning of SNNs composed of Leaky
Integrate and Fire (LIF) neurons with exponentially decayed synapses and soft
reset. The algorithm not only learns the synaptic weight but also adapts the
temporal filters associated to the synapses. Compared to the BPTT algorithm,
SOLSA has much lower memory requirement and achieves a more balanced temporal
workload distribution. Moreover, SOLSA incorporates enhancement techniques such
as scheduled weight update, early stop training and adaptive synapse filter,
which speed up the convergence and enhance the learning performance. When
compared to other non-BPTT based SNN learning, SOLSA demonstrates an average
learning accuracy improvement of 14.2%. Furthermore, compared to BPTT, SOLSA
achieves a 5% higher average learning accuracy with a 72% reduction in memory
cost.

针对脉冲神经网络的在线学习问题，本研究提出了适用于在线学习的神经突触自适应空时学习算法（SOLSA）。相比传统的通过时间反向传播（BPTT）算法，SOLSA 在内存需求上更低，具有更平衡的时间工作负载分配，并且通过调度权重更新、提前停止训练和自适应突触滤波等增强技术，提高了收敛速度和学习性能。与其他非 BPTT 基于的脉冲神经网络学习方法相比，SOLSA 在平均学习准确度上表现出 14.2% 的提升。此外，SOLSA 相比于 BPTT，在平均学习准确度提高了 5% 的同时，内存成本减少了 72%。

基于前向时间轴的神经形态学在线学习空时模式

Neuromorphic Online Learning for Spatiotemporal Patterns with a  Forward-only Timeline

Distributed online learning is gaining increased traction due to its unique
ability to process large-scale datasets and streaming data. To address the
growing public awareness and concern on privacy protection, plenty of private
distributed online learning algorithms have been proposed, mostly based on
differential privacy which has emerged as the ``gold standard" for privacy
protection. However, these algorithms often face the dilemma of trading
learning accuracy for privacy. By exploiting the unique characteristics of
online learning, this paper proposes an approach that tackles the dilemma and
ensures both differential privacy and learning accuracy in distributed online
learning. More specifically, while ensuring a diminishing expected
instantaneous regret, the approach can simultaneously ensure a finite
cumulative privacy budget, even on the infinite time horizon. To cater for the
fully distributed setting, we adopt the local differential-privacy framework
which avoids the reliance on a trusted data curator, and hence, provides
stronger protection than the classic ``centralized" (global) differential
privacy. To the best of our knowledge, this is the first algorithm that
successfully ensures both rigorous local differential privacy and learning
accuracy. The effectiveness of the proposed algorithm is evaluated using
machine learning tasks, including logistic regression on the ``Mushrooms" and
``Covtype" datasets and CNN based image classification on the ``MNIST" and
``CIFAR-10" datasets.

该论文提出了一种针对分布式在线学习的隐私保护算法，在保证微分隐私的同时提高算法的学习准确率，并采用本地微分隐私算法框架来避免对数据收集方的信任，有效地对包括逻辑回归和基于 CNN 的图像分类等任务进行了测试。