Unlike opaque object, novel view synthesis of transparent object is a
challenging task, because transparent object refracts light of background
causing visual distortions on the transparent object surface along the
viewpoint change. Recently introduced Neural Radiance Fields (NeRF) is a view
synthesis method. Thanks to its remarkable performance improvement, lots of
following applications based on NeRF in various topics have been developed.
However, if an object with a different refractive index is included in a scene
such as transparent object, NeRF shows limited performance because refracted
light ray at the surface of the transparent object is not appropriately
considered. To resolve the problem, we propose a NeRF-based method consisting
of the following three steps: First, we reconstruct a three-dimensional shape
of a transparent object using visual hull. Second, we simulate the refraction
of the rays inside of the transparent object according to Snell's law. Last, we
sample points through refracted rays and put them into NeRF. Experimental
evaluation results demonstrate that our method addresses the limitation of
conventional NeRF with transparent objects.

我们提出了一种基于 NeRF 的方法，通过三个步骤解决了传统 NeRF 在透明对象方面的局限性。首先，使用可视外形重建透明对象的三维形状。其次，根据斯涅尔定律模拟透明对象内部的光线折射。最后，通过折射光线采样点，并将其放入 NeRF 中。实验结果表明，我们的方法解决了传统 NeRF 在处理透明对象方面的限制。

透明物体的神经辐射场法使用视体

Neural Radiance Fields for Transparent Object Using Visual Hull

Transparent objects are common in our daily life and frequently handled in
the automated production line. Robust vision-based robotic grasping and
manipulation for these objects would be beneficial for automation. However, the
majority of current grasping algorithms would fail in this case since they
heavily rely on the depth image, while ordinary depth sensors usually fail to
produce accurate depth information for transparent objects owing to the
reflection and refraction of light. In this work, we address this issue by
contributing a large-scale real-world dataset for transparent object depth
completion, which contains 57,715 RGB-D images from 130 different scenes. Our
dataset is the first large-scale, real-world dataset that provides ground truth
depth, surface normals, transparent masks in diverse and cluttered scenes.
Cross-domain experiments show that our dataset is more general and can enable
better generalization ability for models. Moreover, we propose an end-to-end
depth completion network, which takes the RGB image and the inaccurate depth
map as inputs and outputs a refined depth map. Experiments demonstrate superior
efficacy, efficiency and robustness of our method over previous works, and it
is able to process images of high resolutions under limited hardware resources.
Real robot experiments show that our method can also be applied to novel
transparent object grasping robustly. The full dataset and our method are
publicly available at www.graspnet.net/transcg

本研究提供了一种基于深度学习网络的方法，针对普通深度传感器获取透明物体深度信息的难题，构建了一个大规模真实场景数据集，同时提出了一种深度完成网络用于输出精度更高的透明物体深度信息，并且该方法能够成功地应用于机器人对透明物体的抓取。