Learning 3D scene representation from a single-view image is a long-standing
fundamental problem in computer vision, with the inherent ambiguity in
predicting contents unseen from the input view. Built on the recently proposed
3D Gaussian Splatting (3DGS), the Splatter Image method has made promising
progress on fast single-image novel view synthesis via learning a single 3D
Gaussian for each pixel based on the U-Net feature map of an input image.
However, it has limited expressive power to represent occluded components that
are not observable in the input view. To address this problem, this paper
presents a Hierarchical Splatter Image method in which a pixel is worth more
than one 3D Gaussians. Specifically,
each pixel is represented by a parent 3D Gaussian and a small number of child
3D Gaussians. Parent 3D Gaussians are learned as done in the vanilla Splatter
Image. Child 3D Gaussians are learned via a lightweight Multi-Layer Perceptron
(MLP) which takes as input the projected image features of a parent 3D Gaussian
and the embedding of a target camera view. Both parent and child 3D Gaussians
are learned end-to-end in a stage-wise way. The joint condition of input image
features from eyes of the parent Gaussians and the target camera position
facilitates learning to allocate child Gaussians to ``see the unseen'',
recovering the occluded details that are often missed by parent Gaussians.
In experiments, the proposed method is tested on the ShapeNet-SRN and CO3D
datasets with state-of-the-art performance obtained, especially showing
promising capabilities of reconstructing occluded contents in the input view.

从单视图图像中学习 3D 场景表示是计算机视觉中长期存在的一个基本问题，具有从输入视图预测未见内容的固有模糊性。本文提出了一种分层 Splatter 图像方法，通过一种像素由一个父 3D 高斯和少量的子 3D 高斯来表示，解决了典型 Splatter 图像无法表示不能在输入视图中观察到的遮挡部分的问题。

单视角三维重建中，一个像素价值高于三个 3D 高斯函数

A Pixel Is Worth More Than One 3D Gaussians in Single-View 3D  Reconstruction

Monocular 3D detection (M3D) aims for precise 3D object localization from a
single-view image which usually involves labor-intensive annotation of 3D
detection boxes. Weakly supervised M3D has recently been studied to obviate the
3D annotation process by leveraging many existing 2D annotations, but it often
requires extra training data such as LiDAR point clouds or multi-view images
which greatly degrades its applicability and usability in various applications.
We propose SKD-WM3D, a weakly supervised monocular 3D detection framework that
exploits depth information to achieve M3D with a single-view image exclusively
without any 3D annotations or other training data. One key design in SKD-WM3D
is a self-knowledge distillation framework, which transforms image features
into 3D-like representations by fusing depth information and effectively
mitigates the inherent depth ambiguity in monocular scenarios with little
computational overhead in inference. In addition, we design an
uncertainty-aware distillation loss and a gradient-targeted transfer modulation
strategy which facilitate knowledge acquisition and knowledge transfer,
respectively. Extensive experiments show that SKD-WM3D surpasses the
state-of-the-art clearly and is even on par with many fully supervised methods.

利用深度信息，无需任何三维标注或其他训练数据，SKD-WM3D 提出了一种弱监督的单目三维检测框架，通过融合深度信息将图像特征转化为三维表示，有效地消除了单目场景中的深度歧义。实验证明，SKD-WM3D 明显超过了最先进的方法，甚至与许多全监督方法相媲美。