Recent advances in 3D scene representation and novel view synthesis have
witnessed the rise of Neural Radiance Fields (NeRFs). Nevertheless, it is not
trivial to exploit NeRF for the photorealistic 3D scene stylization task, which
aims to generate visually consistent and photorealistic stylized scenes from
novel views. Simply coupling NeRF with photorealistic style transfer (PST) will
result in cross-view inconsistency and degradation of stylized view syntheses.
Through a thorough analysis, we demonstrate that this non-trivial task can be
simplified in a new light: When transforming the appearance representation of a
pre-trained NeRF with Lipschitz mapping, the consistency and photorealism
across source views will be seamlessly encoded into the syntheses. That
motivates us to build a concise and flexible learning framework namely LipRF,
which upgrades arbitrary 2D PST methods with Lipschitz mapping tailored for the
3D scene. Technically, LipRF first pre-trains a radiance field to reconstruct
the 3D scene, and then emulates the style on each view by 2D PST as the prior
to learn a Lipschitz network to stylize the pre-trained appearance. In view of
that Lipschitz condition highly impacts the expressivity of the neural network,
we devise an adaptive regularization to balance the reconstruction and
stylization. A gradual gradient aggregation strategy is further introduced to
optimize LipRF in a cost-efficient manner. We conduct extensive experiments to
show the high quality and robust performance of LipRF on both photorealistic 3D
stylization and object appearance editing.

提出了一种称为 LipRF 的学习框架，该框架利用 Lipschitz 映射，升级任意 2D PST 方法用于 3D 场景，并在光线场重建和 2D PST 的先验下学习 Lipschitz 网络以进行预训练外观的风格化，实验证明 LipRF 在照片逼真 3D 美学和物体外观编辑方面具有高质量和鲁棒性能。