In real-world applications, image degeneration caused by adverse weather is
always complex and changes with different weather conditions from days and
seasons. Systems in real-world environments constantly encounter adverse
weather conditions that are not previously observed. Therefore, it practically
requires adverse weather removal models to continually learn from incrementally
collected data reflecting various degeneration types. Existing adverse weather
removal approaches, for either single or multiple adverse weathers, are mainly
designed for a static learning paradigm, which assumes that the data of all
types of degenerations to handle can be finely collected at one time before a
single-phase learning process. They thus cannot directly handle the incremental
learning requirements. To address this issue, we made the earliest effort to
investigate the continual all-in-one adverse weather removal task, in a setting
closer to real-world applications. Specifically, we develop a novel continual
learning framework with effective knowledge replay (KR) on a unified network
structure. Equipped with a principal component projection and an effective
knowledge distillation mechanism, the proposed KR techniques are tailored for
the all-in-one weather removal task. It considers the characteristics of the
image restoration task with multiple degenerations in continual learning, and
the knowledge for different degenerations can be shared and accumulated in the
unified network structure. Extensive experimental results demonstrate the
effectiveness of the proposed method to deal with this challenging task, which
performs competitively to existing dedicated or joint training image
restoration methods. Our code is available at
this https URL

开发了一种面向实际应用的新型连续学习框架，用于不断学习从不同恶劣天气中提取图像，通过知识回放和共享在统一网络结构中积累了不同恶劣天气的知识。

统一网络结构下的知识重播全方位恶劣天气去除

Continual All-in-One Adverse Weather Removal with Knowledge Replay on a  Unified Network Structure

Removing adverse weather conditions like rain, fog, and snow from images is
an important problem in many applications. Most methods proposed in the
literature have been designed to deal with just removing one type of
degradation. Recently, a CNN-based method using neural architecture search
(All-in-One) was proposed to remove all the weather conditions at once.
However, it has a large number of parameters as it uses multiple encoders to
cater to each weather removal task and still has scope for improvement in its
performance. In this work, we focus on developing an efficient solution for the
all adverse weather removal problem. To this end, we propose TransWeather, a
transformer-based end-to-end model with just a single encoder and a decoder
that can restore an image degraded by any weather condition. Specifically, we
utilize a novel transformer encoder using intra-patch transformer blocks to
enhance attention inside the patches to effectively remove smaller weather
degradations. We also introduce a transformer decoder with learnable weather
type embeddings to adjust to the weather degradation at hand. TransWeather
achieves improvements across multiple test datasets over both All-in-One
network as well as methods fine-tuned for specific tasks. TransWeather is also
validated on real world test images and found to be more effective than
previous methods. Implementation code can be accessed at
this https URL .

本研究提出了一种基于 Transformer 的端到端模型 TransWeather，只使用一个编码器和一个解码器，可以有效去除任何天气条件下的图像失真，其在多个测试数据集上均取得了比 All-in-One 方法和 Fine-tuned 方法更好的效果，并在现实世界的测试图像上进行了验证。