Images from outdoor scenes may be taken under various weather conditions. It
is well studied that weather impacts the performance of computer vision
algorithms and needs to be handled properly. However, existing algorithms model
weather condition as a discrete status and estimate it using multi-label
classification. The fact is that, physically, specifically in meteorology,
weather are modeled as a continuous and transitional status. Instead of
directly implementing hard classification as existing multi-weather
classification methods do, we consider the physical formulation of
multi-weather conditions and model the impact of physical-related parameter on
learning from the image appearance. In this paper, we start with solid revisit
of the physics definition of weather and how it can be described as a
continuous machine learning and computer vision task. Namely, we propose to
model the weather uncertainty, where the level of probability and co-existence
of multiple weather conditions are both considered. A Gaussian mixture model is
used to encapsulate the weather uncertainty and a uncertainty-aware
multi-weather learning scheme is proposed based on prior-posterior learning. A
novel multi-weather co-presence estimation transformer (MeFormer) is proposed.
In addition, a new multi-weather co-presence estimation (MePe) dataset, along
with 14 fine-grained weather categories and 16,078 samples, is proposed to
benchmark both conventional multi-label weather classification task and
multi-weather co-presence estimation task. Large scale experiments show that
the proposed method achieves state-of-the-art performance and substantial
generalization capabilities on both the conventional multi-label weather
classification task and the proposed multi-weather co-presence estimation task.
Besides, modeling weather uncertainty also benefits adverse-weather semantic
segmentation.

本研究提出了一种新的多天气共存估计方法，基于物理原理模型化了天气不确定性，并通过高斯混合模型和先验 - 后验学习实现了不确定性感知的多天气学习方案。实验结果表明该方法在常规多标签天气分类和多天气共存估计任务中均具有最先进的性能和广泛的泛化能力。

多气候共存估计的天气不确定性建模

Modeling Weather Uncertainty for Multi-weather Co-Presence Estimation

In the classical supervised learning settings, classifiers are fit with the
assumption of balanced label distributions and produce remarkable results on
the same. In the real world, however, these assumptions often bend and in turn
adversely impact model performance. Identifying bad learners in skewed target
distributions is even more challenging. Thus achieving model robustness under
these "label shift" settings is an important task in autonomous perception. In
this paper, we analyze the impact of label shift on the task of multi-weather
classification for autonomous vehicles. We use this information as a prior to
better assess pedestrian detection in adverse weather. We model the
classification performance as an indicator of robustness under 4 label shift
scenarios and study the behavior of multiple classes of models. We propose
t-RAIN a similarity mapping technique for synthetic data augmentation using
large scale generative models and evaluate the performance on DAWN dataset.
This mapping boosts model test accuracy by 2.1, 4.4, 1.9, 2.7 % in no-shift,
fog, snow, dust shifts respectively. We present state-of-the-art pedestrian
detection results on real and synthetic weather domains with best performing
82.69 AP (snow) and 62.31 AP (fog) respectively.

通过分析标签偏移对自动驾驶中的多气候分类任务及行人检测任务的影响，并提出使用 t-RAIN 进行数据增广的方法，使得在无偏移、雾、雪和尘埃偏移情形下行人检测的性能提高 2.1％-4.4％，在真实和合成气象领域中分别获得最优的 82.69 AP（雪）和 62.31 AP（雾）的性能。