Most existing style transfer methods follow the assumption that styles can be
represented with global statistics (e.g., Gram matrices or covariance
matrices), and thus address the problem by forcing the output and style images
to have similar global statistics. An alternative is the assumption of local
style patterns, where algorithms are designed to swap similar local features of
content and style images. However, the limitation of these existing methods is
that they neglect the semantic structure of the content image which may lead to
corrupted content structure in the output. In this paper, we make a new
assumption that image features from the same semantic region form a manifold
and an image with multiple semantic regions follows a multi-manifold
distribution. Based on this assumption, the style transfer problem is
formulated as aligning two multi-manifold distributions and a Manifold
Alignment based Style Transfer (MAST) framework is proposed. The proposed
framework allows semantically similar regions between the output and the style
image share similar style patterns. Moreover, the proposed manifold alignment
method is flexible to allow user editing or using semantic segmentation maps as
guidance for style transfer. To allow the method to be applicable to
photorealistic style transfer, we propose a new adaptive weight skip connection
network structure to preserve the content details. Extensive experiments verify
the effectiveness of the proposed framework for both artistic and
photorealistic style transfer. Code is available at
this https URL

本文提出了基于流形对齐的样式转换方法（MAST），通过假设图像的特征分布符合多重流形分布的假设，可以学习到图像的语义区域与样式模式的匹配关系，通过自适应权重跳跃连接网络结构来保留内容的细节，从而实现艺术风格和真实感风格的转换。