Time series foundation models have shown impressive performance on a variety of tasks, across a wide range of domains, even in zero-shot settings. However, most of these models are designed to handle short univariate time series as an input. This limits their practical use, especially in domains such as healthcare with copious amounts of long and multivariate data with strong temporal and intra-variate dependencies. Our study bridges this gap by cataloging and systematically comparing various context expansion techniques from both language and time series domains, and introducing a novel compressive memory mechanism to allow encoder-only TSFMs to effectively model intra-variate dependencies. We demonstrate the benefits of our approach by imbuing MOMENT, a recent family of multi-task time series foundation models, with the multivariate context.

本研究解决了当前时间序列基础模型在处理长时间序列和多变量数据时的局限性。通过对语言和时间序列领域的上下文扩展技术进行系统比较，提出了一种新颖的压缩记忆机制，使得仅使用编码器的时间序列基础模型能够有效建模内部变量间的依赖关系。我们的研究表明，这种方法显著提升了模型在处理长时间序列时的性能，尤其在医疗等领域的应用具有重要影响。

面向长时间序列的基础模型