Multivariate time series forecasting has recently gained great success with
the rapid growth of deep learning models. However, existing approaches usually
train models from scratch using limited temporal data, preventing their
generalization. Recently, with the surge of the Large Language Models (LLMs),
several works have attempted to introduce LLMs into time series forecasting.
Despite promising results, these methods directly take time series as the input
to LLMs, ignoring the inherent modality gap between temporal and text data. In
this work, we propose a novel Large Language Models and time series alignment
framework, dubbed LLaTA, to fully unleash the potentials of LLMs in the time
series forecasting challenge. Based on cross-modal knowledge distillation, the
proposed method exploits both input-agnostic static knowledge and
input-dependent dynamic knowledge in pre-trained LLMs. In this way, it empowers
the forecasting model with favorable performance as well as strong
generalization abilities. Extensive experiments demonstrate the proposed method
establishes a new state of the art for both long- and short-term forecasting.
Code is available at https://github.com/Hank0626/LLaTA.

利用大型语言模型和时间序列对齐框架，进行多元时间序列预测，同时充分发挥语言模型的潜力，建立了长期和短期预测的新的最先进方法。

通过跨模态知识蒸馏来驯服预训练 LLMs 进行广义时间序列预测

Taming Pre-trained LLMs for Generalised Time Series Forecasting via  Cross-modal Knowledge Distillation

Time series alignment methods call for highly expressive, differentiable and
invertible warping functions which preserve temporal topology, i.e
diffeomorphisms. Diffeomorphic warping functions can be generated from the
integration of velocity fields governed by an ordinary differential equation
(ODE). Gradient-based optimization frameworks containing diffeomorphic
transformations require to calculate derivatives to the differential equation's
solution with respect to the model parameters, i.e. sensitivity analysis.
Unfortunately, deep learning frameworks typically lack
automatic-differentiation-compatible sensitivity analysis methods; and implicit
functions, such as the solution of ODE, require particular care. Current
solutions appeal to adjoint sensitivity methods, ad-hoc numerical solvers or
ResNet's Eulerian discretization. In this work, we present a closed-form
expression for the ODE solution and its gradient under continuous
piecewise-affine (CPA) velocity functions. We present a highly optimized
implementation of the results on CPU and GPU. Furthermore, we conduct extensive
experiments on several datasets to validate the generalization ability of our
model to unseen data for time-series joint alignment. Results show significant
improvements both in terms of efficiency and accuracy.

本文提出了基于连续分段仿射速度场的 ODE 解及其梯度的闭式表达式，以实现时间序列联合对齐的高效和准确性。