Neural Radiance Fields (NeRFs) have demonstrated remarkable potential in
capturing complex 3D scenes with high fidelity. However, one persistent
challenge that hinders the widespread adoption of NeRFs is the computational
bottleneck due to the volumetric rendering. On the other hand, 3D Gaussian
splatting (3DGS) has recently emerged as an alternative representation that
leverages a 3D Gaussisan-based representation and adopts the rasterization
pipeline to render the images rather than volumetric rendering, achieving very
fast rendering speed and promising image quality. However, a significant
drawback arises as 3DGS entails a substantial number of 3D Gaussians to
maintain the high fidelity of the rendered images, which requires a large
amount of memory and storage. To address this critical issue, we place a
specific emphasis on two key objectives: reducing the number of Gaussian points
without sacrificing performance and compressing the Gaussian attributes, such
as view-dependent color and covariance. To this end, we propose a learnable
mask strategy that significantly reduces the number of Gaussians while
preserving high performance. In addition, we propose a compact but effective
representation of view-dependent color by employing a grid-based neural field
rather than relying on spherical harmonics. Finally, we learn codebooks to
compactly represent the geometric attributes of Gaussian by vector
quantization. In our extensive experiments, we consistently show over
10$\times$ reduced storage and enhanced rendering speed, while maintaining the
quality of the scene representation, compared to 3DGS. Our work provides a
comprehensive framework for 3D scene representation, achieving high
performance, fast training, compactness, and real-time rendering. Our project
page is available at this https URL

通过学习性的掩码策略和向量量化，我们减少了高斯点的数量和占用的存储空间，实现了高性能、快速训练、紧凑且实时的渲染，提供了一个全面的三维场景表示框架。