Being data-driven is one of the most iconic properties of deep learning
algorithms. The birth of ImageNet drives a remarkable trend of "learning from
large-scale data" in computer vision. Pretraining on ImageNet to obtain rich
universal representations has been manifested to benefit various 2D visual
tasks, and becomes a standard in 2D vision. However, due to the laborious
collection of real-world 3D data, there is yet no generic dataset serving as a
counterpart of ImageNet in 3D vision, thus how such a dataset can impact the 3D
community is unraveled. To remedy this defect, we introduce MVImgNet, a
large-scale dataset of multi-view images, which is highly convenient to gain by
shooting videos of real-world objects in human daily life. It contains 6.5
million frames from 219,188 videos crossing objects from 238 classes, with rich
annotations of object masks, camera parameters, and point clouds. The
multi-view attribute endows our dataset with 3D-aware signals, making it a soft
bridge between 2D and 3D vision.
We conduct pilot studies for probing the potential of MVImgNet on a variety
of 3D and 2D visual tasks, including radiance field reconstruction, multi-view
stereo, and view-consistent image understanding, where MVImgNet demonstrates
promising performance, remaining lots of possibilities for future explorations.
Besides, via dense reconstruction on MVImgNet, a 3D object point cloud
dataset is derived, called MVPNet, covering 87,200 samples from 150 categories,
with the class label on each point cloud. Experiments show that MVPNet can
benefit the real-world 3D object classification while posing new challenges to
point cloud understanding.
MVImgNet and MVPNet will be publicly available, hoping to inspire the broader
vision community.

提出了一个名为 MVImgNet 的大规模多视图图像数据集，它包含了从 238 个类别的 219,188 个视频中捕获的 650 万个帧。这个数据集被用于在 3D 视觉和 2D 视觉任务中进行了各种实验，包括辐射场重建、多视图立体和视角一致的图像理解。此外，MVImgNet 还产生了一个 3D 物体点云数据集 MVPNet，其中每个点云有类别标签，并且有望在 3D 物体分类方面发挥作用。