Augmented Reality (AR) applications necessitates methods of inserting needed
objects into scenes captured by cameras in a way that is coherent with the
surroundings. Common AR applications require the insertion of predefined 3D
objects with known properties and shape. This simplifies the problem since it
is reduced to extracting an illumination model for the object in that scene by
understanding the surrounding light sources. However, it is often not the case
that we have information about the properties of an object, especially when we
depart from a single source image. Our method renders such source fragments in
a coherent way with the target surroundings using only these two images. Our
pipeline uses a Deep Image Prior (DIP) network based on a U-Net architecture as
the main renderer, alongside robust-feature extracting networks that are used
to apply needed losses. Our method does not require any pair-labeled data, and
no extensive training on a dataset. We compare our method using qualitative
metrics to the baseline methods such as Cut and Paste, Cut And Paste Neural
Rendering, and Image Harmonization

我们使用深度图像先验网络（DIP）和鲁棒特征提取网络，通过只使用两个图像在目标环境中以一致的方式渲染源片段，实现了在需求背景下插入所需对象的方法。与基准方法（如剪切和粘贴、剪切和粘贴神经渲染、图像协调等）相比，我们的方法不需要配对标记的数据和大量训练数据，通过定性指标进行了比较。

RPNR: 鲁棒感知神经胶片重新上色

RPNR: Robust-Perception Neural Reshading

Robust face reconstruction from monocular image in general lighting
conditions is challenging. Methods combining deep neural network encoders with
differentiable rendering have opened up the path for very fast monocular
reconstruction of geometry, lighting and reflectance. They can also be trained
in self-supervised manner for increased robustness and better generalization.
However, their differentiable rasterization based image formation models, as
well as underlying scene parameterization, limit them to Lambertian face
reflectance and to poor shape details. More recently, ray tracing was
introduced for monocular face reconstruction within a classic
optimization-based framework and enables state-of-the art results. However
optimization-based approaches are inherently slow and lack robustness. In this
paper, we build our work on the aforementioned approaches and propose a new
method that greatly improves reconstruction quality and robustness in general
scenes. We achieve this by combining a CNN encoder with a differentiable ray
tracer, which enables us to base the reconstruction on much more advanced
personalized diffuse and specular albedos, a more sophisticated illumination
model and a plausible representation of self-shadows. This enables to take a
big leap forward in reconstruction quality of shape, appearance and lighting
even in scenes with difficult illumination. With consistent face attributes
reconstruction, our method leads to practical applications such as relighting
and self-shadows removal. Compared to state-of-the-art methods, our results
show improved accuracy and validity of the approach.

本文提出了一种基于 CNN 编码器和可微分光线追踪器的面部重建方法，使得在不同照明场景下，可以得到更高质量更真实的面部形状、外观和照明，从而实现实用的应用，如重新照明和自动去除自阴影。