We introduce GRM, a large-scale reconstructor capable of recovering a 3D
asset from sparse-view images in around 0.1s. GRM is a feed-forward
transformer-based model that efficiently incorporates multi-view information to
translate the input pixels into pixel-aligned Gaussians, which are unprojected
to create a set of densely distributed 3D Gaussians representing a scene.
Together, our transformer architecture and the use of 3D Gaussians unlock a
scalable and efficient reconstruction framework. Extensive experimental results
demonstrate the superiority of our method over alternatives regarding both
reconstruction quality and efficiency. We also showcase the potential of GRM in
generative tasks, i.e., text-to-3D and image-to-3D, by integrating it with
existing multi-view diffusion models. Our project website is at:
this https URL

我们介绍了 GRM，一个能够从稀疏视图图像中在大约 0.1 秒内恢复 3D 资产的大规模重构器。GRM 是一个前馈变换器模型，能够高效地将多视图信息结合起来，将输入像素翻译成像素对齐的高斯分布，通过反投影生成一系列密集分布的 3D 高斯分布来表示场景。我们的变换器架构和使用 3D 高斯分布的方法共同构成了一个可扩展且高效的重构框架。广泛的实验结果证明，我们的方法在重构质量和效率方面优于其他替代方法。我们还展示了 GRM 在生成任务中的潜力，即将其与现有的多视图扩散模型集成，如文本到 3D 和图像到 3D。有关我们的项目网站，请访问：this https URL

GRM：高效的大型高斯重建模型用于 3D 重建和生成

GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction  and Generation

Novelty detection is the task of recognizing samples that do not belong to
the distribution of the target class. During training, the novelty class is
absent, preventing the use of traditional classification approaches. Deep
autoencoders have been widely used as a base of many unsupervised novelty
detection methods. In particular, context autoencoders have been successful in
the novelty detection task because of the more effective representations they
learn by reconstructing original images from randomly masked images. However, a
significant drawback of context autoencoders is that random masking fails to
consistently cover important structures of the input image, leading to
suboptimal representations - especially for the novelty detection task. In this
paper, to optimize input masking, we have designed a framework consisting of
two competing networks, a Mask Module and a Reconstructor. The Mask Module is a
convolutional autoencoder that learns to generate optimal masks that cover the
most important parts of images. Alternatively, the Reconstructor is a
convolutional encoder-decoder that aims to reconstruct unperturbed images from
masked images. The networks are trained in an adversarial manner in which the
Mask Module generates masks that are applied to images given to the
Reconstructor. In this way, the Mask Module seeks to maximize the
reconstruction error that the Reconstructor is minimizing. When applied to
novelty detection, the proposed approach learns semantically richer
representations compared to context autoencoders and enhances novelty detection
at test time through more optimal masking. Novelty detection experiments on the
MNIST and CIFAR-10 image datasets demonstrate the proposed approach's
superiority over cutting-edge methods. In a further experiment on the UCSD
video dataset for novelty detection, the proposed approach achieves
state-of-the-art results.

研究了使用新型的自编码器方法来提高新颖性检测，使用 Mask Module 生成最优的遮罩，通过两个竞争的网络来优化输入遮罩，达到了比传统方法更好的效果。