Fused Magnesium Furnace (FMF) is a crucial industrial equipment in the
production of magnesia, and anomaly detection plays a pivotal role in ensuring
its efficient, stable, and secure operation. Existing anomaly detection methods
primarily focus on analyzing dominant anomalies using the process variables
(such as arc current) or constructing neural networks based on abnormal visual
features, while overlooking the intrinsic correlation of cross-modal
information. This paper proposes a cross-modal Transformer (dubbed FmFormer),
designed to facilitate anomaly detection in fused magnesium smelting processes
by exploring the correlation between visual features (video) and process
variables (current). Our approach introduces a novel tokenization paradigm to
effectively bridge the substantial dimensionality gap between the 3D video
modality and the 1D current modality in a multiscale manner, enabling a
hierarchical reconstruction of pixel-level anomaly detection. Subsequently, the
FmFormer leverages self-attention to learn internal features within each
modality and bidirectional cross-attention to capture correlations across
modalities. To validate the effectiveness of the proposed method, we also
present a pioneering cross-modal benchmark of the fused magnesium smelting
process, featuring synchronously acquired video and current data for over 2.2
million samples. Leveraging cross-modal learning, the proposed FmFormer
achieves state-of-the-art performance in detecting anomalies, particularly
under extreme interferences such as current fluctuations and visual occlusion
caused by heavy water mist. The presented methodology and benchmark may be
applicable to other industrial applications with some amendments. The benchmark
will be released at this https URL

本文提出了一种跨模态 Transformer（称为 FmFormer），旨在通过探索视觉特征（视频）和过程变量（电流）之间的相关性，促进熔镁冶炼过程中的异常检测。该方法引入了一种新颖的标记化范式，以通过多尺度方式有效地弥合三维视频模态和一维电流模态之间的实质性维度差距，实现逐层的像素级异常检测。随后，FmFormer 利用自注意力机制学习每个模态内部特征，并利用双向交叉注意力机制捕捉跨模态的相关性。通过使用跨模态学习，提出的 FmFormer 在检测异常方面取得了最先进的性能，特别是在电流波动和大量雾气遮挡造成的极端干扰下。所提出的方法和基准还可能适用于其他工业应用，并通过该链接发布了基准数据。