Recent studies of multi-modal Industrial Anomaly Detection (IAD) based on
point clouds and RGB images indicated the importance of exploiting redundancy
and complementarity among modalities for accurate classification and
segmentation. However, achieving multi-modal IAD in practical production lines
remains a work in progress that requires consideration of the trade-offs
between costs and benefits associated with introducing new modalities, while
ensuring compatibility with current processes. Combining fast in-line
inspections with high-resolution, time-consuming, near-line characterization
techniques to enhance detection accuracy fits well into the existing quality
control process, but only part of the samples can be tested with expensive
near-line methods. Thus, the model must have the ability to leverage
multi-modal training and handle incomplete modalities during inference. One
solution is generating cross-modal hallucination to transfer knowledge among
modalities for missing modality issues. In this paper, we propose CMDIAD, a
Cross-Modal Distillation framework for IAD to demonstrate the feasibility of
Multi-modal Training, Few-modal Inference pipeline. Moreover, we investigate
reasons behind the asymmetric performance improvement using point clouds or RGB
images as main modality of inference. This lays the foundation of our future
multi-modal dataset construction for efficient IAD from manufacturing
scenarios.

提出了 CMDIAD 框架用于多模态工业异常检测，并通过生成跨模态幻觉来解决缺失模态的问题，探究了以点云或 RGB 图像作为主要推理模态时的性能改善原因，为制造场景中的高效异常检测奠定了基础。

跨模态蒸馏在工业异常检测中的应用：高效多模态 IAD 的探索

Cross-Modal Distillation in Industrial Anomaly Detection: Exploring  Efficient Multi-Modal IAD

This paper presents a novel framework for robust 3D object detection from
point clouds via cross-modal hallucination. Our proposed approach is agnostic
to either hallucination direction between LiDAR and 4D radar. We introduce
multiple alignments on both spatial and feature levels to achieve simultaneous
backbone refinement and hallucination generation. Specifically, spatial
alignment is proposed to deal with the geometry discrepancy for better instance
matching between LiDAR and radar. The feature alignment step further bridges
the intrinsic attribute gap between the sensing modalities and stabilizes the
training. The trained object detection models can deal with difficult detection
cases better, even though only single-modal data is used as the input during
the inference stage. Extensive experiments on the View-of-Delft (VoD) dataset
show that our proposed method outperforms the state-of-the-art (SOTA) methods
for both radar and LiDAR object detection while maintaining competitive
efficiency in runtime.

通过跨模态幻觉实现稳健的点云三维物体检测的新框架，结合空间和特征的多次对齐以实现骨干网络的细化和幻觉生成，提出了空间对齐和特征对齐的步骤以处理几何差异和感知模态间的属性差距，通过单模态数据进行推断阶段的输入，实现更好的困难检测和高效率的检测。在 View-of-Delft (VoD) 数据集上的广泛实验表明，该方法在雷达和 LiDAR 物体检测上优于最先进的方法，同时保持了竞争性的运行效率。