In this paper, we propose an Instant Photorealistic Style Transfer (IPST)
approach, designed to achieve instant photorealistic style transfer on
super-resolution inputs without the need for pre-training on pair-wise datasets
or imposing extra constraints. Our method utilizes a lightweight StyleNet to
enable style transfer from a style image to a content image while preserving
non-color information. To further enhance the style transfer process, we
introduce an instance-adaptive optimization to prioritize the photorealism of
outputs and accelerate the convergence of the style network, leading to a rapid
training completion within seconds. Moreover, IPST is well-suited for
multi-frame style transfer tasks, as it retains temporal and multi-view
consistency of the multi-frame inputs such as video and Neural Radiance Field
(NeRF). Experimental results demonstrate that IPST requires less GPU memory
usage, offers faster multi-frame transfer speed, and generates photorealistic
outputs, making it a promising solution for various photorealistic transfer
applications.

该论文提出了一种即时的、逼真的风格迁移方法，用于超分辨率输入的风格迁移，无需与成对数据集进行预训练或强加额外约束。通过使用一个轻量级的 StyleNet，我们的方法在保留非颜色信息的同时，实现了从风格图像到内容图像的风格迁移。为了进一步增强风格迁移过程，我们引入了一种实例自适应优化，以优先考虑输出的逼真性并加快风格网络的收敛，从而在几秒钟内快速完成训练。此外，该方法非常适用于多帧风格迁移任务，因为它保持了多帧输入（如视频和神经辐射场）的时间和多视图一致性。实验结果表明，该方法需要更少的 GPU 内存使用，提供更快的多帧迁移速度，并生成逼真的输出，为各种逼真的迁移应用提供了有希望的解决方案。