Automatic optical inspection (AOI) plays a pivotal role in the manufacturing
process, predominantly leveraging high-resolution imaging instruments for
scanning purposes. It detects anomalies by analyzing image textures or
patterns, making it an essential tool in industrial manufacturing and quality
control. Despite its importance, the deployment of models for AOI often faces
challenges. These include limited sample sizes, which hinder effective feature
learning, variations among source domains, and sensitivities to changes in
lighting and camera positions during imaging. These factors collectively
compromise the accuracy of model predictions. Traditional AOI often fails to
capitalize on the rich mechanism-parameter information from machines or inside
images, including statistical parameters, which typically benefit AOI
classification. To address this, we introduce an external modality-guided data
mining framework, primarily rooted in optical character recognition (OCR), to
extract statistical features from images as a second modality to enhance
performance, termed OANet (Ocr-Aoi-Net). A key aspect of our approach is the
alignment of external modality features, extracted using a single
modality-aware model, with image features encoded by a convolutional neural
network. This synergy enables a more refined fusion of semantic representations
from different modalities. We further introduce feature refinement and a gating
function in our OANet to optimize the combination of these features, enhancing
inference and decision-making capabilities. Experimental outcomes show that our
methodology considerably boosts the recall rate of the defect detection model
and maintains high robustness even in challenging scenarios.

通过基于 OCR 的外部模态引导的数据挖掘框架，我们引入一种名为 OANet（Ocr-Aoi-Net）的方法来提取图像的统计特征作为第二个模态，以提升自动光学检测（AOI）模型的性能，并通过特征的精细融合和优化来提高推理和决策能力，在挑战性场景中明显提高缺陷检测模型的召回率并保持高鲁棒性。