We introduce pixelSplat, a feed-forward model that learns to reconstruct 3D radiance fields parameterized by 3D Gaussian primitives from pairs of images. Our model features real-time and memory-efficient rendering for scalable training as well as fast 3D reconstruction at inference time. To overcome local minima inherent to sparse and locally supported representations, we predict a dense probability distribution over 3D and sample Gaussian means from that probability distribution. We make this sampling operation differentiable via a reparameterization trick, allowing us to back-propagate gradients through the Gaussian splatting representation. We benchmark our method on wide-baseline novel view synthesis on the real-world RealEstate10k and ACID datasets, where we outperform state-of-the-art light field transformers and accelerate rendering by 2.5 orders of magnitude while reconstructing an interpretable and editable 3D radiance field.

我们介绍了pixelSplat，这是一个前馈模型，可以从图像对中学习重建由3D高斯基元参数化的3D辐射场。我们的模型具有实时和内存高效的渲染，可进行可扩展训练和快速3D重建。为了克服稀疏和局部表示固有的局部最小值问题，我们预测了3D上的密集概率分布，并从该概率分布中采样高斯均值。我们通过参数化技巧使采样操作可微分，从而使得我们能够通过高斯平铺表示进行梯度反向传播。我们在现实世界的RealEstate10k和ACID数据集上对我们的方法进行了广泛的基线新视角合成基准测试，表明我们在重建可解释和可编辑的3D辐射场时，胜过现有最先进的光场转换器，并且加快了渲染速度2.5个数量级。

pixelSplat: 基于图像对的3D高斯扩散颗粒的可扩展通用3D重建