The continual learning (CL) paradigm aims to enable neural networks to learn
tasks continually in a sequential fashion. The fundamental challenge in this
learning paradigm is catastrophic forgetting previously learned tasks when the
model is optimized for a new task, especially when their data is not
accessible. Current architectural-based methods aim at alleviating the
catastrophic forgetting problem but at the expense of expanding the capacity of
the model. Regularization-based methods maintain a fixed model capacity;
however, previous studies showed the huge performance degradation of these
methods when the task identity is not available during inference (e.g. class
incremental learning scenario). In this work, we propose a novel
architectural-based method referred as SpaceNet for class incremental learning
scenario where we utilize the available fixed capacity of the model
intelligently. SpaceNet trains sparse deep neural networks from scratch in an
adaptive way that compresses the sparse connections of each task in a compact
number of neurons. The adaptive training of the sparse connections results in
sparse representations that reduce the interference between the tasks.
Experimental results show the robustness of our proposed method against
catastrophic forgetting old tasks and the efficiency of SpaceNet in utilizing
the available capacity of the model, leaving space for more tasks to be
learned. In particular, when SpaceNet is tested on the well-known benchmarks
for CL: split MNIST, split Fashion-MNIST, and CIFAR-10/100, it outperforms
regularization-based methods by a big performance gap. Moreover, it achieves
better performance than architectural-based methods without model expansion and
achieved comparable results with rehearsal-based methods, while offering a huge
memory reduction.

在类增量学习场景中，利用可用模型容量提出 SpaceNet，该方法从头开始以自适应方式训练稀疏的深度神经网络，压缩每个任务的稀疏连接到较少的神经元，实现稀疏表示并降低任务之间的干扰。在多个 CL 基准测试中，SpaceNet 优于基于正则化的方法，并且比扩容的基于架构的方法性能更好，同时还实现了大幅度的内存缩减。

SpaceNet: 为持续学习腾出自由空间

SpaceNet: Make Free Space For Continual Learning

Machine learning has proven to be useful in classification and segmentation
of images. In this paper, we evaluate a training methodology for pixel-wise
segmentation on high resolution satellite images using progressive growing of
generative adversarial networks. We apply our model to segmenting building
rooftops and compare these results to conventional methods for rooftop
segmentation. We present our findings using the SpaceNet version 2 dataset.
Progressive GAN training achieved a test accuracy of 93% compared to 89% for
traditional GAN training.

本研究使用递进增长的生成对抗网络对高分辨率卫星图像进行像素级分割，以分割建筑物屋顶。通过在 SpaceNet 版本 2 数据集上进行测试，相较于传统 GAN 训练，递进 GAN 训练获得了更高的测试准确性，达到了 93%。

逐步增长的生成对抗网络在卫星影像高分辨率语义分割中的应用

Progressively Growing Generative Adversarial Networks for High  Resolution Semantic Segmentation of Satellite Images

Foundational mapping remains a challenge in many parts of the world,
particularly in dynamic scenarios such as natural disasters when timely updates
are critical. Updating maps is currently a highly manual process requiring a
large number of human labelers to either create features or rigorously validate
automated outputs. We propose that the frequent revisits of earth imaging
satellite constellations may accelerate existing efforts to quickly update
foundational maps when combined with advanced machine learning techniques.
Accordingly, the SpaceNet partners (CosmiQ Works, Radiant Solutions, and
NVIDIA), released a large corpus of labeled satellite imagery on Amazon Web
Services (AWS) called SpaceNet. The SpaceNet partners also launched a series of
public prize competitions to encourage improvement of remote sensing machine
learning algorithms. The first two of these competitions focused on automated
building footprint extraction, and the most recent challenge focused on road
network extraction. In this paper we discuss the SpaceNet imagery, labels,
evaluation metrics, prize challenge results to date, and future plans for the
SpaceNet challenge series.

利用地球成像卫星阵列进行快速更新基础地图的先进机器学习技术，结合 SpaceNet 的图像标签与公共奖项竞赛加速影像判读自动化的建筑与道路网络提取，以解决人工标记不足的问题。