Satellite image time series in the optical and infrared spectrum suffer from
frequent data gaps due to cloud cover, cloud shadows, and temporary sensor
outages. It has been a long-standing problem of remote sensing research how to
best reconstruct the missing pixel values and obtain complete, cloud-free image
sequences. We approach that problem from the perspective of representation
learning and develop U-TILISE, an efficient neural model that is able to
implicitly capture spatio-temporal patterns of the spectral intensities, and
that can therefore be trained to map a cloud-masked input sequence to a
cloud-free output sequence. The model consists of a convolutional spatial
encoder that maps each individual frame of the input sequence to a latent
encoding; an attention-based temporal encoder that captures dependencies
between those per-frame encodings and lets them exchange information along the
time dimension; and a convolutional spatial decoder that decodes the latent
embeddings back into multi-spectral images. We experimentally evaluate the
proposed model on EarthNet2021, a dataset of Sentinel-2 time series acquired
all over Europe, and demonstrate its superior ability to reconstruct the
missing pixels. Compared to a standard interpolation baseline, it increases the
PSNR by 1.8 dB at previously seen locations and by 1.3 dB at unseen locations.

利用表示学习的思路，发展了 U-TILISE，一种高效的神经模型，能够隐式地捕捉光谱强度的时空模式，并通过训练将云层遮掩的输入序列映射为无云的输出序列。实验结果表明，与标准插值基准相比，在之前看到的位置，PSNR 增加了 1.8 dB，在未见过的位置增加了 1.3 dB。