To improve the robustness to rain, we present a physically-based rain rendering pipeline for realistically inserting rain into clear weather images. Our rendering relies on a physical particle simulator, an estimation of the scene lighting and an accurate rain photometric modeling to augment images with arbitrary amount of realistic rain or fog. We validate our rendering with a user study, proving our rain is judged 40% more realistic that state-of-the-art. Using our generated weather augmented Kitti and Cityscapes dataset, we conduct a thorough evaluation of deep object detection and semantic segmentation algorithms and show that their performance decreases in degraded weather, on the order of 15% for object detection and 60% for semantic segmentation. Furthermore, we show refining existing networks with our augmented images improves the robustness of both object detection and semantic segmentation algorithms. We experiment on nuScenes and measure an improvement of 15% for object detection and 35% for semantic segmentation compared to original rainy performance. Augmented databases and code are available on the project page.

提出了基于物理原理的雨渲染技术，通过物理粒子模拟器和精确的雨光度建模实现向晴朗天气图像中添加任意数量逼真雨或雾的效果。通过用户研究验证了渲染效果比已有技术更加逼真。利用渲染后生成的天气增强数据进行了深度物体检测和语义分割算法的全面评估，并表明它们的性能在恶劣天气下会有所降低，通过使用增强数据可以提高算法的稳健性。在 nuScenes 数据集上进行了实验，结果显示，与原始的降雨性能相比，物体检测的性能提高了 15％，语义分割的性能提高了 35％。

基于物理的渲染以提高对雨的鲁棒性