The vision community has started to build with the recently developed state
space model, Mamba, as the new backbone for a range of tasks. This paper shows
that Mamba's visual capability can be significantly enhanced through
autoregressive pretraining, a direction not previously explored.
Efficiency-wise, the autoregressive nature can well capitalize on the Mamba's
unidirectional recurrent structure, enabling faster overall training speed
compared to other training strategies like mask modeling. Performance-wise,
autoregressive pretraining equips the Mamba architecture with markedly higher
accuracy over its supervised-trained counterparts and, more importantly,
successfully unlocks its scaling potential to large and even huge model sizes.
For example, with autoregressive pretraining, a base-size Mamba attains 83.2\%
ImageNet accuracy, outperforming its supervised counterpart by 2.0\%; our
huge-size Mamba, the largest Vision Mamba to date, attains 85.0\% ImageNet
accuracy (85.5\% when finetuned with $384\times384$ inputs), notably surpassing
all other Mamba variants in vision. The code is available at
https://github.com/OliverRensu/ARM.

Mamba 的视觉能力可以通过自回归预训练显著增强，而且可以成功解锁其大规模模型尺寸的潜力。

用 Mamba 在视觉中进行自回归预训练

Autoregressive Pretraining with Mamba in Vision

Within today's large-scale systems, one anomaly can impact millions of users.
Detecting such events in real-time is essential to maintain the quality of
services. It allows the monitoring team to prevent or diminish the impact of a
failure. Logs are a core part of software development and maintenance, by
recording detailed information at runtime. Such log data are universally
available in nearly all computer systems. They enable developers as well as
system maintainers to monitor and dissect anomalous events. For Cloud computing
companies and large online platforms in general, growth is linked to the
scaling potential. Automatizing the anomaly detection process is a promising
way to ensure the scalability of monitoring capacities regarding the increasing
volume of logs generated by modern systems. In this paper, we will introduce
MoniLog, a distributed approach to detect real-time anomalies within
large-scale environments. It aims to detect sequential and quantitative
anomalies within a multi-source log stream. MoniLog is designed to structure a
log stream and perform the monitoring of anomalous sequences. Its output
classifier learns from the administrator's actions to label and evaluate the
criticality level of anomalies.

本文介绍了 MoniLog，一种用于检测大型系统中实时异常的分布式方法，旨在检测多源日志流中的顺序和数量异常，并通过输出分类器从管理员的操作中学习标记和评估异常的重要性。