Diminished reality (DR) refers to the removal of real objects from the
environment by virtually replacing them with their background. Modern DR
frameworks use inpainting to hallucinate unobserved regions. While recent deep
learning-based inpainting is promising, the DR use case is complicated by the
need to generate coherent structure and 3D geometry (i.e., depth), in
particular for advanced applications, such as 3D scene editing. In this paper,
we propose DeepDR, a first RGB-D inpainting framework fulfilling all
requirements of DR: Plausible image and geometry inpainting with coherent
structure, running at real-time frame rates, with minimal temporal artifacts.
Our structure-aware generative network allows us to explicitly condition color
and depth outputs on the scene semantics, overcoming the difficulty of
reconstructing sharp and consistent boundaries in regions with complex
backgrounds. Experimental results show that the proposed framework can
outperform related work qualitatively and quantitatively.

Diminished reality 通过使用 inpainting 技术生成并纠正场景的图像和几何结构，DeepDR 框架在实时帧速率下运行，具有最小的时间伪影，并在复杂背景下重建锐利和一致的边界。