We present Decomposer, a semi-supervised reconstruction model that decomposes
distorted image sequences into their fundamental building blocks - the original
image and the applied augmentations, i.e., shadow, light, and occlusions. To
solve this problem, we use the SIDAR dataset that provides a large number of
distorted image sequences: each sequence contains images with shadows,
lighting, and occlusions applied to an undistorted version. Each distortion
changes the original signal in different ways, e.g., additive or multiplicative
noise. We propose a transformer-based model to explicitly learn this
decomposition. The sequential model uses 3D Swin-Transformers for
spatio-temporal encoding and 3D U-Nets as prediction heads for individual parts
of the decomposition. We demonstrate that by separately pre-training our model
on weakly supervised pseudo labels, we can steer our model to optimize for our
ambiguous problem definition and learn to differentiate between the different
image distortions.

我们提出了一种名为 Decomposer 的半监督重建模型，它将失真的图像序列分解为它们的基本构建块 - 原始图像和应用的增强，例如阴影、光线和遮挡。