We introduce pixelSplat, a feed-forward model that learns to reconstruct 3D
radiance fields parameterized by 3D Gaussian primitives from pairs of images.
Our model features real-time and memory-efficient rendering for scalable
training as well as fast 3D reconstruction at inference time. To overcome local
minima inherent to sparse and locally supported representations, we predict a
dense probability distribution over 3D and sample Gaussian means from that
probability distribution. We make this sampling operation differentiable via a
reparameterization trick, allowing us to back-propagate gradients through the
Gaussian splatting representation. We benchmark our method on wide-baseline
novel view synthesis on the real-world RealEstate10k and ACID datasets, where
we outperform state-of-the-art light field transformers and accelerate
rendering by 2.5 orders of magnitude while reconstructing an interpretable and
editable 3D radiance field.

我们介绍了 pixelSplat，这是一个前馈模型，可以从图像对中学习重建由 3D 高斯基元参数化的 3D 辐射场。我们的模型具有实时和内存高效的渲染，可进行可扩展训练和快速 3D 重建。为了克服稀疏和局部表示固有的局部最小值问题，我们预测了 3D 上的密集概率分布，并从该概率分布中采样高斯均值。我们通过参数化技巧使采样操作可微分，从而使得我们能够通过高斯平铺表示进行梯度反向传播。我们在现实世界的 RealEstate10k 和 ACID 数据集上对我们的方法进行了广泛的基线新视角合成基准测试，表明我们在重建可解释和可编辑的 3D 辐射场时，胜过现有最先进的光场转换器，并且加快了渲染速度 2.5 个数量级。