Current adversarial attacks on motion estimation, or optical flow, optimize small per-pixel perturbations, which are unlikely to appear in the real world. In contrast, adverse weather conditions constitute a much more realistic threat scenario. Hence, in this work, we present a novel attack on motion estimation that exploits adversarially optimized particles to mimic weather effects like snowflakes, rain streaks or fog clouds. At the core of our attack framework is a differentiable particle rendering system that integrates particles (i) consistently over multiple time steps (ii) into the 3D space (iii) with a photo-realistic appearance. Through optimization, we obtain adversarial weather that significantly impacts the motion estimation. Surprisingly, methods that previously showed good robustness towards small per-pixel perturbations are particularly vulnerable to adversarial weather. At the same time, augmenting the training with non-optimized weather increases a method's robustness towards weather effects and improves generalizability at almost no additional cost.

本文提出了一种新的对运动估计的攻击方法，利用对气溶胶进行的对抗优化使运动估计器受到雪花、雨滴、雾云等恶劣天气的影响，通过优化训练不仅可以增强运动估计器对天气干扰的鲁棒性，还可以提高模型的推广能力。

干扰性倾盆大雨：用于动态估计的对抗性天气攻击