We present an approach to infer the 3D shape, texture, and camera pose for an
object from a single RGB image, using only category-level image collections
with foreground masks as supervision. We represent the shape as an
image-conditioned implicit function that transforms the surface of a sphere to
that of the predicted mesh, while additionally predicting the corresponding
texture. To derive supervisory signal for learning, we enforce that: a) our
predictions when rendered should explain the available image evidence, and b)
the inferred 3D structure should be geometrically consistent with learned pixel
to surface mappings. We empirically show that our approach improves over prior
work that leverages similar supervision, and in fact performs competitively to
methods that use stronger supervision. Finally, as our method enables learning
with limited supervision, we qualitatively demonstrate its applicability over a
set of about 30 object categories.

使用有前景蒙版的类别级图像集合进行有限监督学习，推断出单个 RGB 图像的物体的 3D 形状、纹理和相机姿态，并展示其在大约 30 个物体类别上的应用可行性。

无标注图像集的隐式网格重建

Implicit Mesh Reconstruction from Unannotated Image Collections

We study end-to-end learning strategies for 3D shape inference from images,
in particular from a single image. Several approaches in this direction have
been investigated that explore different shape representations and suitable
learning architectures. We focus instead on the underlying probabilistic
mechanisms involved and contribute a more principled probabilistic
inference-based reconstruction framework, which we coin Probabilistic
Reconstruction Networks. This framework expresses image conditioned 3D shape
inference through a family of latent variable models, and naturally decouples
the choice of shape representations from the inference itself. Moreover, it
suggests different options for the image conditioning and allows training in
two regimes, using either Monte Carlo or variational approximation of the
marginal likelihood. Using our Probabilistic Reconstruction Networks we obtain
single image 3D reconstruction results that set a new state of the art on the
ShapeNet dataset in terms of the intersection over union and earth mover's
distance evaluation metrics. Interestingly, we obtain these results using a
basic voxel grid representation, improving over recent work based on finer
point cloud or mesh based representations.

使用基于概率推理的重建框架，探索了基于不同形状表示和适当的学习架构的端到端学习策略，从而实现了从单个图像推断三维形状。通过我们的 Probabilistic Reconstruction Networks，使用基本的体素网格表征，实现了在 ShapeNet 数据集上的新的交集和移动距离评估指标的最新成果。

基于概率重建网络的单张图片 3D 形态推断

Probabilistic Reconstruction Networks for 3D Shape Inference from a  Single Image

We present a learning-based model to infer the personalized 3D shape of
people from a few frames (1-8) of a monocular video in which the person is
moving, in less than 10 seconds with a reconstruction accuracy of 5mm. Our
model learns to predict the parameters of a statistical body model and instance
displacements that add clothing and hair to the shape. The model achieves fast
and accurate predictions based on two key design choices. First, by predicting
shape in a canonical T-pose space, the network learns to encode the images of
the person into pose-invariant latent codes, where the information is fused.
Second, based on the observation that feed-forward predictions are fast but do
not always align with the input images, we predict using both, bottom-up and
top-down streams (one per view) allowing information to flow in both
directions. Learning relies only on synthetic 3D data. Once learned, the model
can take a variable number of frames as input, and is able to reconstruct
shapes even from a single image with an accuracy of 6mm. Results on 3 different
datasets demonstrate the efficacy and accuracy of our approach.

本文提出了一种基于学习的模型，利用少量单目视频帧 (1-8 帧) 从移动的人中推断出其个性化的 3D 形状，可在不到 10 秒内达到 5mm 的重建精度。该模型学习预测一个统计体型模型的参数和实例化偏移量，同时加入衣服和头发的形状，具有快速和准确的预测，这得益于其对 T-pose 空间的预测和从上往下和从下往上两个视图预测的结合。该模型仅基于合成的 3D 数据进行学习，可将可变数量的帧作为输入，即使仅有一张图像，也能以 6mm 的精度重建形状。3 个不同数据集的结果表明了本方法的有效性和准确性。