3D Gaussian Splatting showcases notable advancements in photo-realistic and
real-time novel view synthesis. However, it faces challenges in modeling mirror
reflections, which exhibit substantial appearance variations from different
viewpoints. To tackle this problem, we present MirrorGaussian, the first method
for mirror scene reconstruction with real-time rendering based on 3D Gaussian
Splatting. The key insight is grounded on the mirror symmetry between the
real-world space and the virtual mirror space. We introduce an intuitive
dual-rendering strategy that enables differentiable rasterization of both the
real-world 3D Gaussians and the mirrored counterpart obtained by reflecting the
former about the mirror plane. All 3D Gaussians are jointly optimized with the
mirror plane in an end-to-end framework. MirrorGaussian achieves high-quality
and real-time rendering in scenes with mirrors, empowering scene editing like
adding new mirrors and objects. Comprehensive experiments on multiple datasets
demonstrate that our approach significantly outperforms existing methods,
achieving state-of-the-art results. Project page:
this https URL

借助 3D 高斯透射，利用镜像对称性，MirrorGaussian 提出了第一种基于 3D 高斯透射的实时渲染镜像场景重建方法，通过可微分光栅化策略优化 3D 高斯透射及其映射到镜像平面的过程，实现了高质量和实时渲染，对于带有镜子的场景达到了最先进的效果。

MirrorGaussian：反射 3D 高斯函数用于重建镜面反射

MirrorGaussian: Reflecting 3D Gaussians for Reconstructing Mirror  Reflections

3D Gaussian Splatting (3DGS) has marked a significant breakthrough in the
realm of 3D scene reconstruction and novel view synthesis. However, 3DGS, much
like its predecessor Neural Radiance Fields (NeRF), struggles to accurately
model physical reflections, particularly in mirrors that are ubiquitous in
real-world scenes. This oversight mistakenly perceives reflections as separate
entities that physically exist, resulting in inaccurate reconstructions and
inconsistent reflective properties across varied viewpoints. To address this
pivotal challenge, we introduce Mirror-3DGS, an innovative rendering framework
devised to master the intricacies of mirror geometries and reflections, paving
the way for the generation of realistically depicted mirror reflections. By
ingeniously incorporating mirror attributes into the 3DGS and leveraging the
principle of plane mirror imaging, Mirror-3DGS crafts a mirrored viewpoint to
observe from behind the mirror, enriching the realism of scene renderings.
Extensive assessments, spanning both synthetic and real-world scenes, showcase
our method's ability to render novel views with enhanced fidelity in real-time,
surpassing the state-of-the-art Mirror-NeRF specifically within the challenging
mirror regions. Our code will be made publicly available for reproducible
research.

Mirror-3DGS is an innovative rendering framework that addresses the challenge of accurately modeling mirror reflections within the 3DGS approach, resulting in enhanced fidelity and real-time rendering of novel views specifically within mirror regions.