Depth reconstruction and hyperspectral reflectance reconstruction are two
active research topics in computer vision and image processing. Conventionally,
these two topics have been studied separately using independent imaging setups
and there is no existing method which can acquire depth and spectral
reflectance simultaneously in one shot without using special hardware. In this
paper, we propose a novel single-shot hyperspectral-depth reconstruction method
using an off-the-shelf RGB camera and projector. Our method is based on a
single color-dot projection, which simultaneously acts as structured light for
depth reconstruction and spatially-varying color illuminations for
hyperspectral reflectance reconstruction. To jointly reconstruct the depth and
the hyperspectral reflectance from a single color-dot image, we propose a novel
end-to-end network architecture that effectively incorporates a geometric
color-dot pattern loss and a photometric hyperspectral reflectance loss.
Through the experiments, we demonstrate that our hyperspectral-depth
reconstruction method outperforms the combination of an existing
state-of-the-art single-shot hyperspectral reflectance reconstruction method
and depth reconstruction method.

该论文提出了一种新的单次多光谱深度重建方法，使用 RGB 相机和投影仪，利用单个彩色点图像实现深度重建和空间变化颜色照明的多光谱反射重建，文中介绍了网络架构，并通过实验表明该方法优于现有单次多光谱反射重建方法和深度重建方法的组合。