To enhance predicting performance while minimizing computational demands,
this paper introduces a joint time-frequency domain Transformer (JTFT) for
multivariate forecasting. The method exploits the sparsity of time series in
the frequency domain using a small number of learnable frequencies to extract
temporal dependencies effectively. Alongside the frequency domain
representation, a fixed number of the most recent data points are directly
encoded in the time domain, bolstering the learning of local relationships and
mitigating the adverse effects of non-stationarity. JTFT achieves linear
complexity since the length of the internal representation remains independent
of the input sequence length. Additionally, a low-rank attention layer is
proposed to efficiently capture cross-dimensional dependencies and prevent
performance degradation due to the entanglement of temporal and channel-wise
modeling. Experiments conducted on six real-world datasets demonstrate that
JTFT outperforms state-of-the-art methods.

该研究提出了一种联合时频域变压器模型 (JTFT) 进行多元预测的方法，其中使用少量可学习的频率，利用时间序列在频域中的稀疏性来有效提取时间依赖关系，并且 JTFT 通过在时域中直接编码最近的数据点来增强局部关系的学习能力和减轻不稳定性的不利影响，该方法线性复杂度，同时采用低秩注意层来高效捕获跨维度依赖关系并提高模型性能，实验证明 JTFT 优于现有的方法。