3D pose estimation is a challenging but important task in computer vision. In this work, we show that standard deep learning approaches to 3D pose estimation are not robust when objects are partially occluded or viewed from a previously unseen pose. Inspired by the robustness of generative vision models to partial occlusion, we propose to integrate deep neural networks with 3D generative representations of objects into a unified neural architecture that we term NeMo. In particular, NeMo learns a generative model of neural feature activations at each vertex on a dense 3D mesh. Using differentiable rendering we estimate the 3D object pose by minimizing the reconstruction error between NeMo and the feature representation of the target image. To avoid local optima in the reconstruction loss, we train the feature extractor to maximize the distance between the individual feature representations on the mesh using contrastive learning. Our extensive experiments on PASCAL3D+, occluded-PASCAL3D+ and ObjectNet3D show that NeMo is much more robust to partial occlusion and unseen pose compared to standard deep networks, while retaining competitive performance on regular data. Interestingly, our experiments also show that NeMo performs reasonably well even when the mesh representation only crudely approximates the true object geometry with a cuboid, hence revealing that the detailed 3D geometry is not needed for accurate 3D pose estimation. The code is publicly available at https://github.com/Angtian/NeMo.

本文研究 3D 姿态估计问题中常见的物体部分遮挡或者从未见过的视角所导致的深度学习问题，通过将深度神经网络与物体的三维生成表示相结合，提出了一种名为 NeMo 的神经架构，该方法在不需要真实的 3D 几何体来进行准确的 3D 姿态估计方面具有更强的鲁棒性，并使用对比学习的方法来训练特征提取器以增强其鲁棒性。 在 PASCAL3D +，occluded-PASCAL3D +和ObjectNet3D数据集上的广泛实验证明，NeMo 相对于标准深度网络具有更好的性能和更强的鲁棒性。

NeMo: 鲁棒的三维姿态估计的对比特征的神经网格模型