Log anomaly detection is a key component in the field of artificial
intelligence for IT operations (AIOps). Considering log data of variant
domains, retraining the whole network for unknown domains is inefficient in
real industrial scenarios especially for low-resource domains. However,
previous deep models merely focused on extracting the semantics of log sequence
in the same domain, leading to poor generalization on multi-domain logs.
Therefore, we propose a unified Transformer-based framework for log anomaly
detection (\ourmethod{}), which is comprised of the pretraining and
adapter-based tuning stage. Our model is first pretrained on the source domain
to obtain shared semantic knowledge of log data. Then, we transfer the
pretrained model to the target domain via the adapter-based tuning. The
proposed method is evaluated on three public datasets including one source
domain and two target domains. The experimental results demonstrate that our
simple yet efficient approach, with fewer trainable parameters and lower
training costs in the target domain, achieves state-of-the-art performance on
three benchmarks.

我们提出了一种基于 Transformer 的日志异常检测方法，包括预训练和适配器调整阶段，旨在实现多领域的良好泛化性能，并在一个源域和两个目标域的三个数据集上取得了最先进的性能。

TransLog: 一种基于 Transformer 的统一框架用于日志异常检测

TransLog: A Unified Transformer-based Framework for Log Anomaly Detection

Adapter-based tuning has recently arisen as an alternative to fine-tuning. It
works by adding light-weight adapter modules to a pretrained language model
(PrLM) and only updating the parameters of adapter modules when learning on a
downstream task. As such, it adds only a few trainable parameters per new task,
allowing a high degree of parameter sharing. Prior studies have shown that
adapter-based tuning often achieves comparable results to fine-tuning. However,
existing work only focuses on the parameter-efficient aspect of adapter-based
tuning while lacking further investigation on its effectiveness. In this paper,
we study the latter. We first show that adapter-based tuning better mitigates
forgetting issues than fine-tuning since it yields representations with less
deviation from those generated by the initial PrLM. We then empirically compare
the two tuning methods on several downstream NLP tasks and settings. We
demonstrate that 1) adapter-based tuning outperforms fine-tuning on
low-resource and cross-lingual tasks; 2) it is more robust to overfitting and
less sensitive to changes in learning rates.

本文研究了基于 Adapter 的微调方法。实验证明，对于某些具有挑战性的任务，Adapter-based tuning 方法比 fine-tuning 效果更好，并且更抗过拟合和更不敏感于学习率的变化。