Deep learning model (primarily convolutional networks and LSTM) for time
series classification has been studied broadly by the community with the wide
applications in different domains like healthcare, finance, industrial
engineering and IoT. Meanwhile, Transformer Networks recently achieved frontier
performance on various natural language processing and computer vision tasks.
In this work, we explored a simple extension of the current Transformer
Networks with gating, named Gated Transformer Networks (GTN) for the
multivariate time series classification problem. With the gating that merges
two towers of Transformer which model the channel-wise and step-wise
correlations respectively, we show how GTN is naturally and effectively
suitable for the multivariate time series classification task. We conduct
comprehensive experiments on thirteen dataset with full ablation study. Our
results show that GTN is able to achieve competing results with current
state-of-the-art deep learning models. We also explored the attention map for
the natural interpretability of GTN on time series modeling. Our preliminary
results provide a strong baseline for the Transformer Networks on multivariate
time series classification task and grounds the foundation for future research.

本研究探索了带门控的 Transformer Networks 的时间序列分类问题，我们展示了 GTN 在多元时间序列分类任务上的自然有效性，并且在 13 个数据集上进行了实验证明 GTN 可以达到具有竞争力的结果。我们还探究了 GTN 的注意力图以提高时间序列建模的自然解释性。