Federated Learning (FL) is a promising distributed method for edge-level
machine learning, particularly for privacysensitive applications such as those
in military and medical domains, where client data cannot be shared or
transferred to a cloud computing server. In many use-cases, communication cost
is a major challenge in FL due to its natural intensive network usage. Client
devices, such as smartphones or Internet of Things (IoT) nodes, have limited
resources in terms of energy, computation, and memory. To address these
hardware constraints, lightweight models and compression techniques such as
pruning and quantization are commonly adopted in centralised paradigms. In this
paper, we investigate the impact of compression techniques on FL for a typical
image classification task. Going further, we demonstrate that a straightforward
method can compresses messages up to 50% while having less than 1% of accuracy
loss, competing with state-of-the-art techniques.

本文研究压缩技术对典型图像分类任务的联邦学习的影响，并证明了一种简单的方法可以在保持不到 1% 准确率损失的同时压缩 50% 的消息，与最先进的技术相媲美。

面向轻量通信设计的联邦学习压缩

Federated learning compression designed for lightweight communications

The translational equivariant nature of Convolutional Neural Networks (CNNs)
is a reason for its great success in computer vision. However, networks do not
enjoy more general equivariance properties such as rotation or scaling,
ultimately limiting their generalization performance. To address this
limitation, we devise a method that endows CNNs with simultaneous equivariance
with respect to translation, rotation, and scaling. Our approach defines a
convolution-like operation and ensures equivariance based on our proposed
scalable Fourier-Argand representation. The method maintains similar efficiency
as a traditional network and hardly introduces any additional learnable
parameters, since it does not face the computational issue that often occurs in
group-convolution operators. We validate the efficacy of our approach in the
image classification task, demonstrating its robustness and the generalization
ability to both scaled and rotated inputs.

本文提出了一种方法，通过使用可缩放的傅里叶 - 阿甘德表示法和类似卷积的操作来实现卷积神经网络对于平移、旋转和缩放的同时等变性，并验证了该方法在图像分类任务方面的有效性和对缩放和旋转输入的泛化能力。

利用类似卷积实现网络的尺度和旋转等变性增强

Empowering Networks With Scale and Rotation Equivariance Using A  Similarity Convolution

The brains of all bilaterally symmetric animals on Earth are divided into
left and right hemispheres. The anatomy and functionality of the hemispheres
have a large degree of overlap, but there are asymmetries and they specialize
to possess different attributes. Several studies have used computational models
to mimic hemispheric asymmetries with a focus on reproducing human data on
semantic and visual processing tasks. In this study, we aimed to understand how
dual hemispheres could interact in a given task. We propose a bilateral
artificial neural network that imitates a lateralization observed in nature:
that the left hemisphere specializes in specificity and the right in
generalities. We used two ResNet-9 convolutional neural networks with different
training objectives and tested it on an image classification task. Our analysis
found that the hemispheres represent complementary features that are exploited
by a network head which implements a type of weighted attention. The bilateral
architecture outperformed a range of baselines of similar representational
capacity that don't exploit differential specialization. The results
demonstrate the efficacy of bilateralism, contribute to an understanding of
bilateralism in biological brains and the architecture serves as an inductive
bias when designing new AI systems.

该研究提出了一种双侧神经网络结构，模仿大脑左右脑半球的功能区别，并通过对图像分类任务的测试表明，这种结构比其他容量类似的基线模型更具有效性，可以实现对图像特征的加权注意力。

半球专业化下的双侧大脑深度学习

Deep learning in a bilateral brain with hemispheric specialization

In this paper we investigate the performance of different types of rectified
activation functions in convolutional neural network: standard rectified linear
unit (ReLU), leaky rectified linear unit (Leaky ReLU), parametric rectified
linear unit (PReLU) and a new randomized leaky rectified linear units (RReLU).
We evaluate these activation function on standard image classification task.
Our experiments suggest that incorporating a non-zero slope for negative part
in rectified activation units could consistently improve the results. Thus our
findings are negative on the common belief that sparsity is the key of good
performance in ReLU. Moreover, on small scale dataset, using deterministic
negative slope or learning it are both prone to overfitting. They are not as
effective as using their randomized counterpart. By using RReLU, we achieved
75.68\% accuracy on CIFAR-100 test set without multiple test or ensemble.

本文通过评估不同类型的修正线性单元 rectified activation functions（包括：标准修正线性单元（ReLU），泄漏修正线性单元（Leaky ReLU），参数修正线性单元（PReLU）以及随机泄漏修正线性单元（RReLU））在图像分类任务中的表现，结论表明，对修正激活单元中的负部分引入非零斜率可以始终改善结果，从而推翻了稀疏性是 ReLU 良好性能的关键的常见信念。另外，在小规模数据集上，使用确定性的负斜率或学习固定斜率都容易过拟合，使用随机斜率则更为有效。通过使用 RReLU，我们在 CIFAR-100 测试集上实现了 75.68％的准确度（无多次测试或集合）。