Sleep abnormalities can have severe health consequences. Automated sleep
staging, i.e. labelling the sequence of sleep stages from the patient's
physiological recordings, could simplify the diagnostic process. Previous work
on automated sleep staging has achieved great results, mainly relying on the
EEG signal. However, often multiple sources of information are available beyond
EEG. This can be particularly beneficial when the EEG recordings are noisy or
even missing completely. In this paper, we propose CoRe-Sleep, a Coordinated
Representation multimodal fusion network that is particularly focused on
improving the robustness of signal analysis on imperfect data. We demonstrate
how appropriately handling multimodal information can be the key to achieving
such robustness. CoRe-Sleep tolerates noisy or missing modalities segments,
allowing training on incomplete data. Additionally, it shows state-of-the-art
performance when testing on both multimodal and unimodal data using a single
model on SHHS-1, the largest publicly available study that includes sleep stage
labels. The results indicate that training the model on multimodal data does
positively influence performance when tested on unimodal data. This work aims
at bridging the gap between automated analysis tools and their clinical
utility.

本文提出了 CoRe-Sleep，一种多模态融合网络，专注于改善不完美数据上的信号分析鲁棒性。训练 CoRe-Sleep 可以处理多模态信息，允许在不完整数据上训练。使用单个模型在 SHHS-1 上进行测试时，CoRe-Sleep 显示出多模态数据和单模态数据的最新表现。该工作旨在弥合自动分析工具和临床实用性之间的差距。

CoRe-Sleep：一种多模态融合框架，适用于对非完美模态具有鲁棒性的时间序列

CoRe-Sleep: A Multimodal Fusion Framework for Time Series Robust to  Imperfect Modalities

There has been an increased interest in multimodal language processing
including multimodal dialog, question answering, sentiment analysis, and speech
recognition. However, naturally occurring multimodal data is often imperfect as
a result of imperfect modalities, missing entries or noise corruption. To
address these concerns, we present a regularization method based on tensor rank
minimization. Our method is based on the observation that high-dimensional
multimodal time series data often exhibit correlations across time and
modalities which leads to low-rank tensor representations. However, the
presence of noise or incomplete values breaks these correlations and results in
tensor representations of higher rank. We design a model to learn such tensor
representations and effectively regularize their rank. Experiments on
multimodal language data show that our model achieves good results across
various levels of imperfection.

本文介绍了一种基于张量秩最小化的正则化方法，以处理自然发生的多模态数据中的不完美性，并通过实验在多模态语言数据上实现了良好的结果。