In recent developments, predictive models for multivariate time series
analysis have exhibited commendable performance through the adoption of the
prevalent principle of channel independence. Nevertheless, it is imperative to
acknowledge the intricate interplay among channels, which fundamentally
influences the outcomes of multivariate predictions. Consequently, the notion
of channel independence, while offering utility to a certain extent, becomes
increasingly impractical, leading to information degradation. In response to
this pressing concern, we present CSformer, an innovative framework
characterized by a meticulously engineered two-stage self-attention mechanism.
This mechanism is purposefully designed to enable the segregated extraction of
sequence-specific and channel-specific information, while sharing parameters to
promote synergy and mutual reinforcement between sequences and channels.
Simultaneously, we introduce sequence adapters and channel adapters, ensuring
the model's ability to discern salient features across various dimensions.
Rigorous experimentation, spanning multiple real-world datasets, underscores
the robustness of our approach, consistently establishing its position at the
forefront of predictive performance across all datasets. This augmentation
substantially enhances the capacity for feature extraction inherent to
multivariate time series data, facilitating a more comprehensive exploitation
of the available information.

我们介绍了 CSformer 模型，它通过精心设计的两阶段自注意机制实现了序列特定和通道特定信息的提取，并引入了序列适配器和通道适配器，以促进各个维度之间的协同和相互增强，从而显著提高了多变量时间序列数据的特征提取能力。

通道和序列的舞动：一种高效的基于注意力的多元时间序列预测方法

Dance of Channel and Sequence: An Efficient Attention-Based Approach for  Multivariate Time Series Forecasting

Multivariate time series (MTS) analysis prevails in real-world applications
such as finance, climate science and healthcare. The various self-attention
mechanisms, the backbone of the state-of-the-art Transformer-based models,
efficiently discover the temporal dependencies, yet cannot well capture the
intricate cross-correlation between different features of MTS data, which
inherently stems from complex dynamical systems in practice. To this end, we
propose a novel correlated attention mechanism, which not only efficiently
captures feature-wise dependencies, but can also be seamlessly integrated
within the encoder blocks of existing well-known Transformers to gain
efficiency improvement. In particular, correlated attention operates across
feature channels to compute cross-covariance matrices between queries and keys
with different lag values, and selectively aggregate representations at the
sub-series level. This architecture facilitates automated discovery and
representation learning of not only instantaneous but also lagged
cross-correlations, while inherently capturing time series auto-correlation.
When combined with prevalent Transformer baselines, correlated attention
mechanism constitutes a better alternative for encoder-only architectures,
which are suitable for a wide range of tasks including imputation, anomaly
detection and classification. Extensive experiments on the aforementioned tasks
consistently underscore the advantages of correlated attention mechanism in
enhancing base Transformer models, and demonstrate our state-of-the-art results
in imputation, anomaly detection and classification.

我们提出了一种新颖的相关注意力机制，能够高效地捕捉多元时间序列数据中不同特征之间的复杂相互关系，并能够与现有的基于 Transformer 的模型无缝集成，提高效率。在多种任务中，包括插补、异常检测和分类，相关注意力机制与先前的 Transformer 模型相结合，形成了更好的编码器架构，并且取得了先进的结果。

变量时间序列变压器中的关联注意力

Correlated Attention in Transformers for Multivariate Time Series

Identifying causal relations among simultaneously acquired signals is an
important problem in multivariate time series analysis. For linear stochastic
systems Granger proposed a simple procedure called the Granger causality to
detect such relations. In this work we consider nonlinear extensions of
Granger's idea and refer to the result as Extended Granger Causality. A simple
approach implementing the Extended Granger Causality is presented and applied
to multiple chaotic time series and other types of nonlinear signals. In
addition, for situations with three or more time series we propose a
conditional Extended Granger Causality measure that enables us to determine
whether the causal relation between two signals is direct or mediated by
another process.

本文提出了扩展格兰杰因果关系方法，应用于多个混沌时间序列和其他非线性信号，同时在三个或更多时间序列的情况下提出了条件扩展格兰杰因果关系度量。