Can we capture shape and reflectance in stealth? Such capability would be
valuable for many application domains in vision, xR, robotics, and HCI. We
introduce Structured Polarization, the first depth and reflectance sensing
method using patterns of polarized light (SPIDeRS). The key idea is to modulate
the angle of linear polarization (AoLP) of projected light at each pixel. The
use of polarization makes it invisible and lets us recover not only depth but
also directly surface normals and even reflectance. We implement SPIDeRS with a
liquid crystal spatial light modulator (SLM) and a polarimetric camera. We
derive a novel method for robustly extracting the projected structured
polarization pattern from the polarimetric object appearance. We evaluate the
effectiveness of SPIDeRS by applying it to a number of real-world objects. The
results show that our method successfully reconstructs object shapes of various
materials and is robust to diffuse reflection and ambient light. We also
demonstrate relighting using recovered surface normals and reflectance. We
believe SPIDeRS opens a new avenue of polarization use in visual sensing.

通过使用极化光的模式，我们引入了结构极化技术 (SPIDeRS)，首个深度和反射感应方法，为视觉、虚拟现实、机器人和人机交互等应用领域提供了捕捉形状和反射能力。我们的方法使用极化光角度的调制，实现了不仅深度，而且直接获得表面法线和反射信息的能力。通过实验验证，我们的方法成功重建了不同材料对象的形状，并且对漫反射和环境光具备鲁棒性，同时还展示了使用恢复的表面法线和反射信息进行重照明的能力。我们相信 SPIDeRS 开辟了极化应用在视觉感应领域的新途径。

SPIDeRS：结构化偏振用于隐形深度和反射率感知

SPIDeRS: Structured Polarization for Invisible Depth and Reflectance  Sensing

This paper proposes the first self-supervised 6D object pose prediction from
multimodal RGB+polarimetric images. The novel training paradigm comprises 1) a
physical model to extract geometric information of polarized light, 2) a
teacher-student knowledge distillation scheme and 3) a self-supervised loss
formulation through differentiable rendering and an invertible physical
constraint. Both networks leverage the physical properties of polarized light
to learn robust geometric representations by encoding shape priors and
polarization characteristics derived from our physical model. Geometric
pseudo-labels from the teacher support the student network without the need for
annotated real data. Dense appearance and geometric information of objects are
obtained through a differentiable renderer with the predicted pose for
self-supervised direct coupling. The student network additionally features our
proposed invertible formulation of the physical shape priors that enables
end-to-end self-supervised training through physical constraints of derived
polarization characteristics compared against polarimetric input images. We
specifically focus on photometrically challenging objects with texture-less or
reflective surfaces and transparent materials for which the most prominent
performance gain is reported.

本研究提出了第一个自监督的多模态 RGB + 偏振图像的 6D 目标姿态预测方法，通过物理模型、师生知识蒸馏和逆可逆的物理约束等方式实现了稳健的几何表征和对偏振光的特征编码。通过自监督训练和物理约束，得到了准确的物体外观和几何信息，特别适用于具有挑战性纹理、反光表面和透明材料的物体。