Understanding the 3D world is a fundamental problem in computer vision.
However, learning a good representation of 3D objects is still an open problem
due to the high dimensionality of the data and many factors of variation
involved. In this work, we investigate the task of single-view 3D object
reconstruction from a learning agent's perspective. We formulate the learning
process as an interaction between 3D and 2D representations and propose an
encoder-decoder network with a novel projection loss defined by the perspective
transformation. More importantly, the projection loss enables the unsupervised
learning using 2D observation without explicit 3D supervision. We demonstrate
the ability of the model in generating 3D volume from a single 2D image with
three sets of experiments: (1) learning from single-class objects; (2) learning
from multi-class objects and (3) testing on novel object classes. Results show
superior performance and better generalization ability for 3D object
reconstruction when the projection loss is involved.

从学习代理的角度探究了单视角 3D 物体重建任务，提出了一种包含透视变换定义的新型投影损失的编码器 - 解码器网络，实现了从单个 2D 图像生成 3D 体积的无监督学习，并通过实验证明了投影损失提高了 3D 对象重建的性能和泛化能力。

透视变换网络：学习单视图三维物体重建，无需三维监督

Perspective Transformer Nets: Learning Single-View 3D Object  Reconstruction without 3D Supervision

We develop a deep learning algorithm for contour detection with a fully
convolutional encoder-decoder network. Different from previous low-level edge
detection, our algorithm focuses on detecting higher-level object contours. Our
network is trained end-to-end on PASCAL VOC with refined ground truth from
inaccurate polygon annotations, yielding much higher precision in object
contour detection than previous methods. We find that the learned model
generalizes well to unseen object classes from the same super-categories on MS
COCO and can match state-of-the-art edge detection on BSDS500 with fine-tuning.
By combining with the multiscale combinatorial grouping algorithm, our method
can generate high-quality segmented object proposals, which significantly
advance the state-of-the-art on PASCAL VOC (improving average recall from 0.62
to 0.67) with a relatively small amount of candidates ($\sim$1660 per image).

该论文基于深度学习的算法，使用全卷积编码器 - 解码器网络来检测对象轮廓并生成高质量分割对象建议，通过在 PASCAL VOC 和 MS COCO 上进行训练和测试，该算法在较少的候选数情况下相对于基准方法有更高的准确度。