Accurately predicting the 3D shape of any arbitrary object in any pose from a
single image is a key goal of computer vision research. This is challenging as
it requires a model to learn a representation that can infer both the visible
and occluded portions of any object using a limited training set. A training
set that covers all possible object shapes is inherently infeasible. Such
learning-based approaches are inherently vulnerable to overfitting, and
successfully implementing them is a function of both the architecture design
and the training approach. We present an extensive investigation of factors
specific to architecture design, training, experiment design, and evaluation
that influence reconstruction performance and measurement. We show that our
proposed SDFNet achieves state-of-the-art performance on seen and unseen shapes
relative to existing methods GenRe and OccNet. We provide the first large-scale
evaluation of single image shape reconstruction to unseen objects. The source
code, data and trained models can be found on
this https URL

该研究提出了一种基于 SDFNet 的创新方法，可以通过仅仅一张图像实现对物体的三维形状进行准确预测，相对于 GenRe 和 OccNet 方法来说，该方法在对已知和未知物体的形状重建方面实现了最优性能。

单张图像的新型物体形状的三维重建

3D Reconstruction of Novel Object Shapes from Single Images

Rapid advances in 2D perception have led to systems that accurately detect
objects in real-world images. However, these systems make predictions in 2D,
ignoring the 3D structure of the world. Concurrently, advances in 3D shape
prediction have mostly focused on synthetic benchmarks and isolated objects. We
unify advances in these two areas. We propose a system that detects objects in
real-world images and produces a triangle mesh giving the full 3D shape of each
detected object. Our system, called Mesh R-CNN, augments Mask R-CNN with a mesh
prediction branch that outputs meshes with varying topological structure by
first predicting coarse voxel representations which are converted to meshes and
refined with a graph convolution network operating over the mesh's vertices and
edges. We validate our mesh prediction branch on ShapeNet, where we outperform
prior work on single-image shape prediction. We then deploy our full Mesh R-CNN
system on Pix3D, where we jointly detect objects and predict their 3D shapes.

该研究提出了一种称为 Mesh R-CNN 的系统，将 2D 感知和 3D 形状预测相结合，通过预测三角形网格的方式获取检测物体的完整 3D 形状，并在 Pix3D 上应用该系统来共同检测物体和预测它们的 3D 形状。

Mesh R-CNN

We propose an approach to predict the 3D shape and pose for the objects
present in a scene. Existing learning based methods that pursue this goal make
independent predictions per object, and do not leverage the relationships
amongst them. We argue that reasoning about these relationships is crucial, and
present an approach to incorporate these in a 3D prediction framework. In
addition to independent per-object predictions, we predict pairwise relations
in the form of relative 3D pose, and demonstrate that these can be easily
incorporated to improve object level estimates. We report performance across
different datasets (SUNCG, NYUv2), and show that our approach significantly
improves over independent prediction approaches while also outperforming
alternate implicit reasoning methods.

本研究提出了一种基于深度学习的方法，结合独立对象的预测和对象间关系的推理，实现场景中物体的三维形状和姿态预测，并在不同数据集上进行了性能测试。

3D-RelNet：三维预测的联合物体和关系网络

3D-RelNet: Joint Object and Relational Network for 3D Prediction

We propose a method for predicting the 3D shape of a deformable surface from
a single view. By contrast with previous approaches, we do not need a
pre-registered template of the surface, and our method is robust to the lack of
texture and partial occlusions. At the core of our approach is a {\it
geometry-aware} deep architecture that tackles the problem as usually done in
analytic solutions: first perform 2D detection of the mesh and then estimate a
3D shape that is geometrically consistent with the image. We train this
architecture in an end-to-end manner using a large dataset of synthetic
renderings of shapes under different levels of deformation, material
properties, textures and lighting conditions. We evaluate our approach on a
test split of this dataset and available real benchmarks, consistently
improving state-of-the-art solutions with a significantly lower computational
time.

提出了一种基于几何感知的深度学习方法，能够在没有预先注册表面模板的情况下，通过单一视角的输入图像，精确预测可变形表面的 3D 形状， 并采用合成数据集和真实基准数据集的实验证明其在 3D 形状预测上超越同类算法，且计算时间明显减少。