The introduction of large, foundational models to computer vision has led to
drastically improved performance on the task of semantic segmentation. However,
these existing methods exhibit a large performance drop when testing on images
degraded by weather conditions such as rain, fog, or snow. We introduce a
general paired-training method that can be applied to all current foundational
model architectures that leads to improved performance on images in adverse
weather conditions. To this end, we create the WeatherProof Dataset, the first
semantic segmentation dataset with accurate clear and adverse weather image
pairs, which not only enables our new training paradigm, but also improves the
evaluation of the performance gap between clear and degraded segmentation. We
find that training on these paired clear and adverse weather frames which share
an underlying scene results in improved performance on adverse weather data.
With this knowledge, we propose a training pipeline which accentuates the
advantages of paired-data training using consistency losses and language
guidance, which leads to performance improvements by up to 18.4% as compared to
standard training procedures.

使用通用的配对训练方法，改进了大型计算机视觉模型在恶劣气候下的语义分割性能，并创建了 WeatherProof 数据集，提供了准确的恶劣气候和晴朗天气的图像对，进一步改善了分割性能的评估，并通过一种采用一致性损失和语言引导的训练流程，使性能提高了 18.4%。