This paper proposes the first self-supervised 6D object pose prediction from
multimodal RGB+polarimetric images. The novel training paradigm comprises 1) a
physical model to extract geometric information of polarized light, 2) a
teacher-student knowledge distillation scheme and 3) a self-supervised loss
formulation through differentiable rendering and an invertible physical
constraint. Both networks leverage the physical properties of polarized light
to learn robust geometric representations by encoding shape priors and
polarization characteristics derived from our physical model. Geometric
pseudo-labels from the teacher support the student network without the need for
annotated real data. Dense appearance and geometric information of objects are
obtained through a differentiable renderer with the predicted pose for
self-supervised direct coupling. The student network additionally features our
proposed invertible formulation of the physical shape priors that enables
end-to-end self-supervised training through physical constraints of derived
polarization characteristics compared against polarimetric input images. We
specifically focus on photometrically challenging objects with texture-less or
reflective surfaces and transparent materials for which the most prominent
performance gain is reported.

本研究提出了第一个自监督的多模态 RGB + 偏振图像的 6D 目标姿态预测方法，通过物理模型、师生知识蒸馏和逆可逆的物理约束等方式实现了稳健的几何表征和对偏振光的特征编码。通过自监督训练和物理约束，得到了准确的物体外观和几何信息，特别适用于具有挑战性纹理、反光表面和透明材料的物体。