Traditional approaches for learning 3D object categories have been
predominantly trained and evaluated on synthetic datasets due to the
unavailability of real 3D-annotated category-centric data. Our main goal is to
facilitate advances in this field by collecting real-world data in a magnitude
similar to the existing synthetic counterparts. The principal contribution of
this work is thus a large-scale dataset, called Common Objects in 3D, with real
multi-view images of object categories annotated with camera poses and ground
truth 3D point clouds. The dataset contains a total of 1.5 million frames from
nearly 19,000 videos capturing objects from 50 MS-COCO categories and, as such,
it is significantly larger than alternatives both in terms of the number of
categories and objects. We exploit this new dataset to conduct one of the first
large-scale "in-the-wild" evaluations of several new-view-synthesis and
category-centric 3D reconstruction methods. Finally, we contribute NerFormer -
a novel neural rendering method that leverages the powerful Transformer to
reconstruct an object given a small number of its views. The CO3D dataset is
available at this https URL .

该研究提供了一种名为 CO3D 的大规模数据集，其中包含了 50 种 MS-COCO 类别的近 19,000 个视频，共计 1.5 百万帧真实的多视角图像，带有相机姿态和地面真实 3D 点云的注释，并利用该数据集进行了新视角综合和类别为中心的 3D 重建方法的大规模 “野外” 评估，并提出了一种基于 Transformer 的新型神经渲染方法 NerFormer，可根据物体的少数视角重建整个物体。

常见三维物体：大规模学习和评估真实场景下的 3D 类别重建

Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life  3D Category Reconstruction

Traditional approaches for learning 3D object categories use either synthetic
data or manual supervision. In this paper, we propose a method which does not
require manual annotations and is instead cued by observing objects from a
moving vantage point. Our system builds on two innovations: a Siamese viewpoint
factorization network that robustly aligns different videos together without
explicitly comparing 3D shapes; and a 3D shape completion network that can
extract the full shape of an object from partial observations. We also
demonstrate the benefits of configuring networks to perform probabilistic
predictions as well as of geometry-aware data augmentation schemes. We obtain
state-of-the-art results on publicly-available benchmarks.

本篇论文旨在通过两种创新，基于从运动视角观察物体的方式，不需要手动注释，实现学习 3D 物体类别的传统方法。我们的系统基于两种创新：一种是具有鲁棒性的 Siamese 视点因子分解网络，可以对不同的视频进行对齐；另一种是可以从部分观测中提取对象的完整形状的 3D 形状完成网络。我们还演示了配置网络以执行概率预测和几何感知数据增强方案的好处。在公开可用的基准测试中，我们获得了最先进的结果。