In the current data-intensive era, big data has become a significant asset
for Artificial Intelligence (AI), serving as a foundation for developing
data-driven models and providing insight into various unknown fields. This
study navigates through the challenges of data uncertainties, storage
limitations, and predictive data-driven modeling using big data. We utilize
Robust Principal Component Analysis (RPCA) for effective noise reduction and
outlier elimination, and Optimal Sensor Placement (OSP) for efficient data
compression and storage. The proposed OSP technique enables data compression
without substantial information loss while simultaneously reducing storage
needs. While RPCA offers an enhanced alternative to traditional Principal
Component Analysis (PCA) for high-dimensional data management, the scope of
this work extends its utilization, focusing on robust, data-driven modeling
applicable to huge data sets in real-time. For that purpose, Long Short-Term
Memory (LSTM) networks, a type of recurrent neural network, are applied to
model and predict data based on a low-dimensional subset obtained from OSP,
leading to a crucial acceleration of the training phase. LSTMs are feasible for
capturing long-term dependencies in time series data, making them particularly
suited for predicting the future states of physical systems on historical data.
All the presented algorithms are not only theorized but also simulated and
validated using real thermal imaging data mapping a ship's engine.

在当前的数据密集时代，大数据已成为人工智能（AI）的重要资产，为开发基于数据驱动模型和揭示各个未知领域提供基础。本研究通过应用 Robust Principal Component Analysis（RPCA）进行噪声降低和异常值剔除，以及应用 Optimal Sensor Placement（OSP）进行数据压缩和存储，解决数据不确定性、存储限制和数据驱动模型预测的挑战。通过此方法，从 OSP 获得的低维子集基于 Long Short-Term Memory（LSTM）网络进行模型和数据预测，加快训练阶段，适用于实时海量数据集的鲁棒数据驱动建模，特别适合用于基于历史数据预测物理系统的未来状态。所有提出的算法不仅是理论上的，还使用真实热成像数据对船舶发动机进行模拟和验证。