Current arbitrary style transfer models are limited to either image or video
domains. In order to achieve satisfying image and video style transfers, two
different models are inevitably required with separate training processes on
image and video domains, respectively. In this paper, we show that this can be
precluded by introducing UniST, a Unified Style Transfer framework for both
images and videos. At the core of UniST is a domain interaction transformer
(DIT), which first explores context information within the specific domain and
then interacts contextualized domain information for joint learning. In
particular, DIT enables exploration of temporal information from videos for the
image style transfer task and meanwhile allows rich appearance texture from
images for video style transfer, thus leading to mutual benefits. Considering
heavy computation of traditional multi-head self-attention, we present a simple
yet effective axial multi-head self-attention (AMSA) for DIT, which improves
computational efficiency while maintains style transfer performance. To verify
the effectiveness of UniST, we conduct extensive experiments on both image and
video style transfer tasks and show that UniST performs favorably against
state-of-the-art approaches on both tasks. Our code and results will be
released.

本文介绍了一种基于统一风格转移框架 UnST 和领域交互变压器（DIT）的方法，使得同时能实现对图像和视频完成风格转移任务。实验表明 UniST 在图像和视频领域中与现有方法相比表现更佳，通过简单而有效的轴向多头自我关注（AMSA）获得更高的计算效率和风格转移表现。