Structural Health Monitoring (SHM) is a critical task for ensuring the safety
and reliability of civil infrastructures, typically realized on bridges and
viaducts by means of vibration monitoring. In this paper, we propose for the
first time the use of Transformer neural networks, with a Masked Auto-Encoder
architecture, as Foundation Models for SHM. We demonstrate the ability of these
models to learn generalizable representations from multiple large datasets
through self-supervised pre-training, which, coupled with task-specific
fine-tuning, allows them to outperform state-of-the-art traditional methods on
diverse tasks, including Anomaly Detection (AD) and Traffic Load Estimation
(TLE). We then extensively explore model size versus accuracy trade-offs and
experiment with Knowledge Distillation (KD) to improve the performance of
smaller Transformers, enabling their embedding directly into the SHM edge
nodes. We showcase the effectiveness of our foundation models using data from
three operational viaducts. For AD, we achieve a near-perfect 99.9% accuracy
with a monitoring time span of just 15 windows. In contrast, a state-of-the-art
method based on Principal Component Analysis (PCA) obtains its first good
result (95.03% accuracy) only considering 120 windows. On two different TLE
tasks, our models obtain state-of-the-art performance on multiple evaluation
metrics (R$^2$ score, MAE% and MSE%). On the first benchmark, we achieve an
R$^2$ score of 0.97 and 0.85 for light and heavy vehicle traffic, respectively,
while the best previous approach stops at 0.91 and 0.84. On the second one, we
achieve an R$^2$ score of 0.54 versus the 0.10 of the best existing method.

使用 Transformer 神经网络和自监督预训练技术进行结构健康监测，超过传统方法在异常检测和交通负荷估计等任务上的性能，并通过模型大小和精确度的权衡实现在边缘节点嵌入较小的 Transformers。