Time series anomaly detection is crucial for industrial monitoring services
that handle a large volume of data, aiming to ensure reliability and optimize
system performance. Existing methods often require extensive labeled resources
and manual parameter selection, highlighting the need for automation. This
paper proposes a comprehensive framework for automatic parameter optimization
in time series anomaly detection models. The framework introduces three
optimization targets: prediction score, shape score, and sensitivity score,
which can be easily adapted to different model backbones without prior
knowledge or manual labeling efforts. The proposed framework has been
successfully applied online for over six months, serving more than 50,000 time
series every minute. It simplifies the user's experience by requiring only an
expected sensitive value, offering a user-friendly interface, and achieving
desired detection results. Extensive evaluations conducted on public datasets
and comparison with other methods further confirm the effectiveness of the
proposed framework.

该研究论文提出了一个全面的自动参数优化框架，用于时间序列异常检测模型，通过预测分数、形状分数和灵敏度分数三个优化目标，可以适应不同的模型，简化用户的操作，并取得了良好的检测结果。

监控服务中自动单变量时间序列异常检测的优化目标优化

Refining the Optimization Target for Automatic Univariate Time Series  Anomaly Detection in Monitoring Services

Traditional evaluation metrics for classification in natural language
processing such as accuracy and area under the curve fail to differentiate
between models with different predictive behaviors despite their similar
performance metrics. We introduce sensitivity score, a metric that scrutinizes
models' behaviors at the vocabulary level to provide insights into disparities
in their decision-making logic. We assess the sensitivity score on a set of
representative words in the test set using two classifiers trained for hospital
readmission classification with similar performance statistics. Our experiments
compare the decision-making logic of clinicians and classifiers based on rank
correlations of sensitivity scores. The results indicate that the language
model's sensitivity score aligns better with the professionals than the xgboost
classifier on tf-idf embeddings, which suggests that xgboost uses some spurious
features. Overall, this metric offers a novel perspective on assessing models'
robustness by quantifying their discrepancy with professional opinions. Our
code is available on GitHub (this https URL).

本文介绍了一种新的评估模型性能的方法 —— 敏感分数，并通过比较专业医生和机器学习模型敏感分数在词汇级别上的排名相关性来评估其决策逻辑差异。结果表明，敏感分数可以更准确地衡量模型的鲁棒性，该方法可以用于自然语言处理等领域的分类任务的性能评估。