Existing unsupervised methods have addressed the challenges of inconsistent
paired data and tedious acquisition of ground-truth labels in shadow removal
tasks. However, GAN-based training often faces issues such as mode collapse and
unstable optimization. Furthermore, due to the complex mapping between shadow
and shadow-free domains, merely relying on adversarial learning is not enough
to capture the underlying relationship between two domains, resulting in low
quality of the generated images. To address these problems, we propose a
semantic-guided adversarial diffusion framework for self-supervised shadow
removal, which consists of two stages. At first stage a semantic-guided
generative adversarial network (SG-GAN) is proposed to carry out a coarse
result and construct paired synthetic data through a cycle-consistent
structure. Then the coarse result is refined with a diffusion-based restoration
module (DBRM) to enhance the texture details and edge artifact at second stage.
Meanwhile, we propose a multi-modal semantic prompter (MSP) that aids in
extracting accurate semantic information from real images and text, guiding the
shadow removal network to restore images better in SG-GAN. We conduct
experiments on multiple public datasets, and the experimental results
demonstrate the effectiveness of our method.

我们提出了一种基于语义引导的对抗扩散框架，用于自监督的去除阴影，并通过两个阶段的训练来改善生成图像的质量。我们的方法在多个公开数据集上进行实验，并证明了其有效性。

语义引导的对抗扩散模型用于自监督阴影去除

Semantic-guided Adversarial Diffusion Model for Self-supervised Shadow  Removal

In this paper we present S3R-Net, the Self-Supervised Shadow Removal Network.
The two-branch WGAN model achieves self-supervision relying on the
unify-and-adaptphenomenon - it unifies the style of the output data and infers
its characteristics from a database of unaligned shadow-free reference images.
This approach stands in contrast to the large body of supervised frameworks.
S3R-Net also differentiates itself from the few existing self-supervised models
operating in a cycle-consistent manner, as it is a non-cyclic, unidirectional
solution. The proposed framework achieves comparable numerical scores to recent
selfsupervised shadow removal models while exhibiting superior qualitative
performance and keeping the computational cost low.

本文介绍了 S3R-Net，一种自监督阴影去除网络，利用双分支 WGAN 模型实现自监督，并通过数据库中未对齐的无阴影参考图像推断输出数据的特征。与大部分有监督框架和少数现有的循环一致性自监督模型不同，S3R-Net 是单向、非循环的解决方案，同时在保持计算成本低的同时实现了与最新的自监督阴影去除模型相当的数值得分和更好的定性性能。