A major goal in neuroscience is to discover neural data representations that
generalize. This goal is challenged by variability along recording sessions
(e.g. environment), subjects (e.g. varying neural structures), and sensors
(e.g. sensor noise), among others. Recent work has begun to address
generalization across sessions and subjects, but few study robustness to sensor
failure which is highly prevalent in neuroscience experiments. In order to
address these generalizability dimensions we first collect our own
electroencephalography dataset with numerous sessions, subjects, and sensors,
then study two time series models: EEGNet (Lawhern et al., 2018) and TOTEM
(Talukder et al., 2024). EEGNet is a widely used convolutional neural network,
while TOTEM is a discrete time series tokenizer and transformer model. We find
that TOTEM outperforms or matches EEGNet across all generalizability cases.
Finally through analysis of TOTEM's latent codebook we observe that
tokenization enables generalization.

通过对神经科学实验中的时间序列模型和传感器故障的研究，发现离散时间序列标记器和转换模型能够在各种泛化性能情况下优于或与卷积神经网络相匹配，并通过对标记器的潜在编码簿的分析发现标记化能够实现泛化。

传感器失效下的泛化性能：Tokenization + Transformers 实现更健壮的潜空间

Generalizability Under Sensor Failure: Tokenization + Transformers  Enable More Robust Latent Spaces

Due to the incapability of one sensory measurement to provide enough
information for condition monitoring of some complex engineered industrial
mechanisms and also for overcoming the misleading noise of a single sensor,
multiple sensors are installed to improve the condition monitoring of some
industrial equipment. Therefore, an efficient data fusion strategy is demanded.
In this research, we presented a Denoising Multi-Modal Autoencoder with a
unique training strategy based on contrastive learning paradigm, both being
utilized for the first time in the machine health monitoring realm. The
presented approach, which leverages the merits of both supervised and
unsupervised learning, not only achieves excellent performance in fusing
multiple modalities (or views) of data into an enriched common representation
but also takes data fusion to the next level wherein one of the views can be
omitted during inference time with very slight performance reduction, or even
without any reduction at all. The presented methodology enables multi-modal
fault diagnosis systems to perform more robustly in case of sensor failure
occurrence, and one can also intentionally omit one of the sensors (the more
expensive one) in order to build a more cost-effective condition monitoring
system without sacrificing performance for practical purposes. The
effectiveness of the presented methodology is examined on a real-world private
multi-modal dataset gathered under non-laboratory conditions from a complex
engineered mechanism, an inline four-stroke spark-ignition engine, aiming for
spark plug fault diagnosis. This dataset, which contains the accelerometer and
acoustic signals as two modalities, has a very slight amount of fault, and
achieving good performance on such a dataset promises that the presented method
can perform well on other equipment as well.

利用多传感器融合和对比学习范式，提出了一种去噪多模态自编码器训练策略，用于提高复杂工业装备的机器健康监测效果，并可在传感器故障发生时实现更稳健的多模态故障诊断系统，以及构建成本更低、不损失性能的实际用途的条件监测系统。