Time series anomaly detection plays a vital role in a wide range of
applications. Existing methods require training one specific model for each
dataset, which exhibits limited generalization capability across different
target datasets, hindering anomaly detection performance in various scenarios
with scarce training data. Aiming at this problem, we propose constructing a
general time series anomaly detection model, which is pre-trained on extensive
multi-domain datasets and can subsequently apply to a multitude of downstream
scenarios. The significant divergence of time series data across different
domains presents two primary challenges in building such a general model: (1)
meeting the diverse requirements of appropriate information bottlenecks
tailored to different datasets in one unified model, and (2) enabling
distinguishment between multiple normal and abnormal patterns, both are crucial
for effective anomaly detection in various target scenarios. To tackle these
two challenges, we propose a General time series anomaly Detector with Adaptive
Bottlenecks and Dual Adversarial Decoders (DADA), which enables flexible
selection of bottlenecks based on different data and explicitly enhances clear
differentiation between normal and abnormal series. We conduct extensive
experiments on nine target datasets from different domains. After pre-training
on multi-domain data, DADA, serving as a zero-shot anomaly detector for these
datasets, still achieves competitive or even superior results compared to those
models tailored to each specific dataset.

提出了一种基于多领域数据预训练的时间序列异常检测通用模型 DADA，通过灵活选择信息瓶颈和明确区分正常与异常序列，在多个目标数据集上取得了竞争性甚至优秀的结果。

基于自适应瓶颈和双对抗解码器的通用时间序列异常检测器

Towards a General Time Series Anomaly Detector with Adaptive Bottlenecks  and Dual Adversarial Decoders

We explore how continued pre-training on domain-specific corpora influences
large language models, revealing that training on the raw corpora endows the
model with domain knowledge, but drastically hurts its prompting ability for
question answering. Taken inspiration from human learning via reading
comprehension--practice after reading improves the ability to answer questions
based on the learned knowledge--we propose a simple method for transforming raw
corpora into reading comprehension texts. Each raw text is enriched with a
series of tasks related to its content. Our method, highly scalable and
applicable to any pre-training corpora, consistently enhances performance
across various tasks in three different domains: biomedicine, finance, and law.
Notably, our 7B language model achieves competitive performance with
domain-specific models of much larger scales, such as BloombergGPT-50B.
Furthermore, we demonstrate that domain-specific reading comprehension texts
can improve the model's performance even on general benchmarks, showing the
potential to develop a general model across even more domains. Our model, code,
and data will be available at this https URL

我们研究了如何在特定领域的语料库上继续进行预训练，发现在原始语料库上进行训练赋予了模型领域知识，但严重损害了其对问题回答的提示能力。借鉴人类通过阅读理解进行学习的灵感，我们提出了一种简单的方法，将原始语料库转化为阅读理解文本。每个原始文本都会丰富其内容相关的一系列任务。我们的方法可以高度扩展，适用于任何预训练语料库，并在生物医学、金融和法律三个不同领域的各种任务中持续提升性能。值得注意的是，我们的 7B 语言模型在性能上与规模更大的专门领域模型（如 BloombergGPT-50B）相媲美。此外，我们证明领域特定的阅读理解文本甚至可以提高模型在通用基准上的性能，展示了在更多领域开发通用模型的潜力。我们的模型、代码和数据将可在此 https URL 中获取。