Models for monocular shape reconstruction of surfaces with diffuse reflection
-- shape from shading -- ought to produce distributions of outputs, because
there are fundamental mathematical ambiguities of both continuous (e.g.,
bas-relief) and discrete (e.g., convex/concave) varieties which are also
experienced by humans. Yet, the outputs of current models are limited to point
estimates or tight distributions around single modes, which prevent them from
capturing these effects. We introduce a model that reconstructs a multimodal
distribution of shapes from a single shading image, which aligns with the human
experience of multistable perception. We train a small denoising diffusion
process to generate surface normal fields from $16\times 16$ patches of
synthetic images of everyday 3D objects. We deploy this model patch-wise at
multiple scales, with guidance from inter-patch shape consistency constraints.
Despite its relatively small parameter count and predominantly bottom-up
structure, we show that multistable shape explanations emerge from this model
for ''ambiguous'' test images that humans experience as being multistable. At
the same time, the model produces veridical shape estimates for object-like
images that include distinctive occluding contours and appear less ambiguous.
This may inspire new architectures for stochastic 3D shape perception that are
more efficient and better aligned with human experience.

用单个阴影图像重建表面的单眼形状重建模型应该产生输出的分布，因为连续（例如浮雕）和离散（例如凸 / 凹）多样性的数学不确定性，也被人类经历到。然而，当前模型的输出仅限于点估计或围绕单一模式的紧密分布，这阻碍了它们捕捉这些效应。我们引入了一个模型，可以从单个阴影图像中重建形状的多模态分布，并与多稳态感知的人类经验相一致。我们训练了一个小型去噪扩散过程，从每个 $16	imes 16$ 的合成日常 3D 物体图像补丁中生成表面法线场。我们在多个尺度上逐个补丁地部署该模型，并根据补丁之间的形状一致性约束进行引导。尽管参数数量相对较小且结构主要自底向上，但我们证明该模型对于人类体验为多稳态的 “模糊” 测试图像，产生了多稳态形状解释。同时，该模型对包含有特征性遮挡轮廓且显得较少模糊的物体图像也能产生真实的形状估计。这可能为更高效且更符合人类经验的随机 3D 形状感知启发新的架构。

多稳定形状由补丁扩散产生

Multistable Shape from Shading Emerges from Patch Diffusion

The introduction of consumer RGB-D scanners set off a major boost in 3D
computer vision research. Yet, the precision of existing depth scanners is not
accurate enough to recover fine details of a scanned object. While modern
shading based depth refinement methods have been proven to work well with
Lambertian objects, they break down in the presence of specularities. We
present a novel shape from shading framework that addresses this issue and
enhances both diffuse and specular objects' depth profiles. We take advantage
of the built-in monochromatic IR projector and IR images of the RGB-D scanners
and present a lighting model that accounts for the specular regions in the
input image. Using this model, we reconstruct the depth map in real-time. Both
quantitative tests and visual evaluations prove that the proposed method
produces state of the art depth reconstruction results.

通过利用消费者 RGB-D 扫描仪自带的 IR Projector 和 IR 图片，本研究提出了一种新的遮蔽形状框架来增强精度，并在实时环境下重建深度图，该方法在 Lambertian 对象中表现良好，在综合光滑和反射表面的情况下也取得了最先进的深度重建效果。