The default strategy for training single-view Large Reconstruction Models
(LRMs) follows the fully supervised route using large-scale datasets of
synthetic 3D assets or multi-view captures. Although these resources simplify
the training procedure, they are hard to scale up beyond the existing datasets
and they are not necessarily representative of the real distribution of object
shapes. To address these limitations, in this paper, we introduce Real3D, the
first LRM system that can be trained using single-view real-world images.
Real3D introduces a novel self-training framework that can benefit from both
the existing synthetic data and diverse single-view real images. We propose two
unsupervised losses that allow us to supervise LRMs at the pixel- and
semantic-level, even for training examples without ground-truth 3D or novel
views. To further improve performance and scale up the image data, we develop
an automatic data curation approach to collect high-quality examples from
in-the-wild images. Our experiments show that Real3D consistently outperforms
prior work in four diverse evaluation settings that include real and synthetic
data, as well as both in-domain and out-of-domain shapes. Code and model can be
found here: this https URL

该研究介绍了 Real3D，使用单视图真实世界图像训练的第一个大型重建模型系统，通过引入自主训练框架和无监督的损失函数，同时利用现有的合成数据和多样化的单视图真实图像，提高了性能和扩大了图像数据的规模，实验结果表明 Real3D 在不同的评估设置中优于以前的工作。