The long-coveted task of reconstructing 3D geometry from images is still a
standing problem. In this paper, we build on the power of neural networks and
introduce Pix2Vex, a network trained to convert camera-captured images into 3D
geometry. We present a novel differentiable renderer ($DR$) as a forward
validation means during training. Our key insight is that $DR$s produce images
of a particular appearance, different from typical input images. Hence, we
propose adding an image-to-image translation component, converting between
these rendering styles. This translation closes the training loop, while
allowing to use minimal supervision only, without needing any 3D model as
ground truth. Unlike state-of-the-art methods, our $DR$ is $C^\infty$ smooth
and thus does not display any discontinuities at occlusions or dis-occlusions.
Through our novel training scheme, our network can train on different types of
images, where previous work can typically only train on images of a similar
appearance to those rendered by a $DR$.

本文介绍了 Pix2Vex, 一种利用神经网络将图像转换为 3D 几何形状的方法，通过差异可导渲染器和图像翻译组件使网络能够训练不同类型的图像，其中渲染器平滑连续且无割裂。