There have been remarkable successes in computer vision with deep learning.
While such breakthroughs show robust performance, there have still been many
challenges in learning in-depth knowledge, like occlusion or predicting
physical interactions. Although some recent works show the potential of 3D data
in serving such context, it is unclear how we efficiently provide 3D input to
the 2D models due to the misalignment in dimensionality between 2D and 3D. To
leverage the successes of 2D models in predicting self-occlusions, we design
Ray-marching in Camera Space (RiCS), a new method to represent the
self-occlusions of foreground objects in 3D into a 2D self-occlusion map. We
test the effectiveness of our representation on the human image harmonization
task by predicting shading that is coherent with a given background image. Our
experiments demonstrate that our representation map not only allows us to
enhance the image quality but also to model temporally coherent complex shadow
effects compared with the simulation-to-real and harmonization methods, both
quantitatively and qualitatively. We further show that we can significantly
improve the performance of human parts segmentation networks trained on
existing synthetic datasets by enhancing the harmonization quality with our
method.

通过引入 Ray-marching in Camera Space（RiCS）表示前景物体的自遮挡，用于人类图像协调任务中，我们的实验表明通过我们的表示图不仅可以提高图像质量，而且可以定量定性地模拟时间上的协同阴影效果，并通过增强协调质量获得现有合成数据集上的人体部分分割网络的显着性能提升。