We investigate the role of representations and architectures for classifying
3D shapes in terms of their computational efficiency, generalization, and
robustness to adversarial transformations. By varying the number of training
examples and employing cross-modal transfer learning we study the role of
initialization of existing deep architectures for 3D shape classification. Our
analysis shows that multiview methods continue to offer the best generalization
even without pretraining on large labeled image datasets, and even when trained
on simplified inputs such as binary silhouettes. Furthermore, the performance
of voxel-based 3D convolutional networks and point-based architectures can be
improved via cross-modal transfer from image representations. Finally, we
analyze the robustness of 3D shape classifiers to adversarial transformations
and present a novel approach for generating adversarial perturbations of a 3D
shape for multiview classifiers using a differentiable renderer. We find that
point-based networks are more robust to point position perturbations while
voxel-based and multiview networks are easily fooled with the addition of
imperceptible noise to the input.

本研究研究了 3D 形状分类的表示和架构的作用，通过变化训练示例的数量和使用跨模态迁移学习，研究了现有深度架构的初始化对 3D 形状分类的影响，结果表明，多视图方法即使没有在大型标记图像数据集上预训练，甚至在训练简化输入（如二进制轮廓）时，也能提供最好的泛化性能。此外，从图像表示进行跨模态传输可以改善基于体素的 3D 卷积网络和基于点的架构的性能。最后，通过不可区分噪声，研究了 3D 形状分类器对于对抗性变化的鲁棒性，结果发现基于点的网络对于点位置扰动更为稳定，而基于体素和多视图的网络则很容易被输入中的微不足道的噪声所欺骗。