Recent years have witnessed the success of introducing Transformers to time
series forecasting. From a data generation perspective, we illustrate that
existing Transformers are susceptible to distribution shifts driven by temporal
contexts, whether observed or unobserved. Such context-driven distribution
shift (CDS) introduces biases in predictions within specific contexts and poses
challenges for conventional training paradigm. In this paper, we introduce a
universal calibration methodology for the detection and adaptation of CDS with
a trained Transformer model. To this end, we propose a novel CDS detector,
termed the "residual-based CDS detector" or "Reconditionor", which quantifies
the model's vulnerability to CDS by evaluating the mutual information between
prediction residuals and their corresponding contexts. A high Reconditionor
score indicates a severe susceptibility, thereby necessitating model
adaptation. In this circumstance, we put forth a straightforward yet potent
adapter framework for model calibration, termed the "sample-level
contextualized adapter" or "SOLID". This framework involves the curation of a
contextually similar dataset to the provided test sample and the subsequent
fine-tuning of the model's prediction layer with a limited number of steps. Our
theoretical analysis demonstrates that this adaptation strategy is able to
achieve an optimal equilibrium between bias and variance. Notably, our proposed
Reconditionor and SOLID are model-agnostic and readily adaptable to a wide
range of Transformers. Extensive experiments show that SOLID consistently
enhances the performance of current SOTA Transformers on real-world datasets,
especially on cases with substantial CDS detected by the proposed
Reconditionor, thus validate the effectiveness of the calibration approach.

近年来，我们见证了将 Transformers 引入时间序列预测的成功。我们在数据生成的角度上阐述了现有的 Transformers 对于由时间上下文驱动的分布变化容易受到影响，不论是已观测到还是未观测到的。本文介绍了一种普适的校准方法，用于检测和调整训练过的 Transformer 模型中的上下文驱动分布变化。我们提出了一种新颖的 CDS 检测器 ——“基于残差的 CDS 检测器” 或 “Reconditionor”，通过评估预测残差与其对应上下文之间的互信息，量化模型对于 CDS 的脆弱性。高 Reconditionor 分数表示严重易受影响，因此需要模型调整。在这种情况下，我们提出了一个简单但有效的模型校准框架 ——“样本级上下文适配器” 或 “SOLID”。该框架涉及对于提供的测试样本进行上下文相似数据集的策划，并通过有限步骤对模型的预测层进行微调。我们的理论分析表明，该调整策略能够在偏差和方差之间实现最佳平衡。值得注意的是，我们提出的 Reconditionor 和 SOLID 适用于各种 Transformers，并且易于调整。大量实验证明 SOLID 持续增强了当前真实世界数据集上最先进的 Transformers 的性能，特别是在由提出的 Reconditionor 检测到的具有实质性 CDS 的情况下，从而验证了校准方法的有效性。