Recovering the radiometric properties of a scene (i.e., the reflectance,
illumination, and geometry) is a long-sought ability of computer vision that
can provide invaluable information for a wide range of applications.
Deciphering the radiometric ingredients from the appearance of a real-world
scene, as opposed to a single isolated object, is particularly challenging as
it generally consists of various objects with different material compositions
exhibiting complex reflectance and light interactions that are also part of the
illumination. We introduce the first method for radiometric scene decomposition
that handles those intricacies. We use RGB-D images to bootstrap geometry
recovery and simultaneously recover the complex reflectance and natural
illumination while refining the noisy initial geometry and segmenting the scene
into different material regions. Most important, we handle real-world scenes
consisting of multiple objects of unknown materials, which necessitates the
modeling of spatially-varying complex reflectance, natural illumination,
texture, interreflection and shadows. We systematically evaluate the
effectiveness of our method on synthetic scenes and demonstrate its application
to real-world scenes. The results show that rich radiometric information can be
recovered from RGB-D images and demonstrate a new role RGB-D sensors can play
for general scene understanding tasks.

通过利用 RGB-D 图像重建场景的几何和同时恢复复杂反射和自然照明的 radiometric 成分，从而对多个未知材料对象进行空间变化的建模，可以实现从 RGB-D 图像中恢复丰富的 radiometric 信息，并展示 RGB-D 传感器在场景理解任务中的新角色。