Recent studies have shown remarkable success in universal style transfer
which transfers arbitrary visual styles to content images. However, existing
approaches suffer from the aesthetic-unrealistic problem that introduces
disharmonious patterns and evident artifacts, making the results easy to spot
from real paintings. To address this limitation, we propose AesUST, a novel
Aesthetic-enhanced Universal Style Transfer approach that can generate
aesthetically more realistic and pleasing results for arbitrary styles.
Specifically, our approach introduces an aesthetic discriminator to learn the
universal human-delightful aesthetic features from a large corpus of
artist-created paintings. Then, the aesthetic features are incorporated to
enhance the style transfer process via a novel Aesthetic-aware Style-Attention
(AesSA) module. Such an AesSA module enables our AesUST to efficiently and
flexibly integrate the style patterns according to the global aesthetic channel
distribution of the style image and the local semantic spatial distribution of
the content image. Moreover, we also develop a new two-stage transfer training
strategy with two aesthetic regularizations to train our model more
effectively, further improving stylization performance. Extensive experiments
and user studies demonstrate that our approach synthesizes aesthetically more
harmonious and realistic results than state of the art, greatly narrowing the
disparity with real artist-created paintings. Our code is available at
this https URL

本研究提出了一种新的美学增强的通用风格转移方法，通过引入美学鉴别器和美学感知的注意力机制，以及两阶段的训练策略和正则化方法，实现了对任意风格的艺术作品进行更加真实和美学的风格转移。