Distributed deep learning has recently been attracting more attention in
remote sensing (RS) applications due to the challenges posed by the increased
amount of open data that are produced daily by Earth observation programs.
However, the high communication costs of sending model updates among multiple
nodes are a significant bottleneck for scalable distributed learning. Gradient
sparsification has been validated as an effective gradient compression (GC)
technique for reducing communication costs and thus accelerating the training
speed. Existing state-of-the-art gradient sparsification methods are mostly
based on the "larger-absolute-more-important" criterion, ignoring the
importance of small gradients, which is generally observed to affect the
performance. Inspired by informative representation of manifold structures from
neighborhood information, we propose a simple yet effective dynamic gradient
compression scheme leveraging neighborhood statistics indicator for RS image
interpretation, termed RS-DGC. We first enhance the interdependence between
gradients by introducing the gradient neighborhood to reduce the effect of
random noise. The key component of RS-DGC is a Neighborhood Statistical
Indicator (NSI), which can quantify the importance of gradients within a
specified neighborhood on each node to sparsify the local gradients before
gradient transmission in each iteration. Further, a layer-wise dynamic
compression scheme is proposed to track the importance changes of each layer in
real time. Extensive downstream tasks validate the superiority of our method in
terms of intelligent interpretation of RS images. For example, we achieve an
accuracy improvement of 0.51% with more than 50 times communication compression
on the NWPU-RESISC45 dataset using VGG-19 network.

近期，分布式深度学习在遥感应用中引起了更多的关注，其中梯度稀疏化被验证为一种有效压缩梯度的方法，以减少通信成本并加快训练速度。通过引入梯度邻域来增强梯度之间的相互依赖，并使用邻域统计指标（NSI）来定义梯度的重要性，我们提出了一种动态梯度压缩方案（RS-DGC）用于遥感图像解释，该方法在智能解释遥感图像方面优于现有方法。