2D-based Industrial Anomaly Detection has been widely discussed, however,
multimodal industrial anomaly detection based on 3D point clouds and RGB images
still has many untouched fields. Existing multimodal industrial anomaly
detection methods directly concatenate the multimodal features, which leads to
a strong disturbance between features and harms the detection performance. In
this paper, we propose Multi-3D-Memory (M3DM), a novel multimodal anomaly
detection method with hybrid fusion scheme: firstly, we design an unsupervised
feature fusion with patch-wise contrastive learning to encourage the
interaction of different modal features; secondly, we use a decision layer
fusion with multiple memory banks to avoid loss of information and additional
novelty classifiers to make the final decision. We further propose a point
feature alignment operation to better align the point cloud and RGB features.
Extensive experiments show that our multimodal industrial anomaly detection
model outperforms the state-of-the-art (SOTA) methods on both detection and
segmentation precision on MVTec-3D AD dataset. Code is available at
this https URL

本文提出了一种基于 3D 点云和 RGB 图像的新型多模式异常检测方法 M3DM，其中采用了无监督特征融合、决策层融合和点特征对齐等技术，实验表明该方法优于现有的同类方法。