Gram-based and patch-based approaches are two important research lines of image style transfer. Recent diversified Gram-based methods have been able to produce multiple and diverse reasonable solutions for the same content and style inputs. However, as another popular research interest, the diversity of patch-based methods remains challenging due to the stereotyped style swapping process based on nearest patch matching. To resolve this dilemma, in this paper, we dive into the core style swapping process of patch-based style transfer and explore possible ways to diversify it. What stands out is an operation called shifted style normalization (SSN), the most effective and efficient way to empower existing patch-based methods to generate diverse results for arbitrary styles. The key insight is to use an important intuition that neural patches with higher activation values could contribute more to diversity. Theoretical analyses and extensive experiments are conducted to demonstrate the effectiveness of our method, and compared with other possible options and state-of-the-art algorithms, it shows remarkable superiority in both diversity and efficiency.

本文介绍了两种重要的风格迁移研究方法：基于Gram矩阵和基于patch。尽管Gram矩阵方法已经能够以多样化的方式生成不同风格的结果，但patch方法的多样性仍然具有挑战性。为了解决这个难题，作者提出了一种名为DivSwapper的通用、有效的模块，可以轻松地与现有的patch和Gram方法集成，以产生多样化的结果，同时具有更高的多样性、质量和效率。

DivSwapper：面向多样化补丁基风格迁移