Multivariate time-series data are frequently observed in critical care
settings and are typically characterized by sparsity (missing information) and
irregular time intervals. Existing approaches for learning representations in
this domain handle these challenges by either aggregation or imputation of
values, which in-turn suppresses the fine-grained information and adds
undesirable noise/overhead into the machine learning model. To tackle this
problem, we propose a Self-supervised Transformer for Time-Series (STraTS)
model which overcomes these pitfalls by treating time-series as a set of
observation triplets instead of using the standard dense matrix representation.
It employs a novel Continuous Value Embedding technique to encode continuous
time and variable values without the need for discretization. It is composed of
a Transformer component with multi-head attention layers which enable it to
learn contextual triplet embeddings while avoiding the problems of recurrence
and vanishing gradients that occur in recurrent architectures. In addition, to
tackle the problem of limited availability of labeled data (which is typically
observed in many healthcare applications), STraTS utilizes self-supervision by
leveraging unlabeled data to learn better representations by using time-series
forecasting as an auxiliary proxy task. Experiments on real-world multivariate
clinical time-series benchmark datasets demonstrate that STraTS has better
prediction performance than state-of-the-art methods for mortality prediction,
especially when labeled data is limited. Finally, we also present an
interpretable version of STraTS which can identify important measurements in
the time-series data. Our data preprocessing and model implementation codes are
available at this https URL

提出了一种基于自监督变压器的多元时间序列数据建模方法，使用连续值嵌入技术对连续时间和变量值进行编码，避免了聚合或插值旧方法带来的问题，并利用自监督学习处理有限标记数据情况下的预测问题。在真实的多元临床时间序列基准数据集上进行了实验证明该方法的正确性。