Recently, large language models (LLMs) have evolved into interactive agents,
proficient in planning, tool use, and task execution across a wide variety of
tasks. However, without specific agent tuning, open-source models like LLaMA
currently struggle to match the efficiency of GPT- 4, particularly given the
scarcity of agent-tuning datasets for fine-tuning. In response, we introduce
\textsc{Mimir}: a streamlined platform offering a customizable pipeline that
enables users to leverage both private knowledge and publicly available,
legally compliant datasets at scale for \textbf{personalized agent tuning}.
Additionally, \textsc{Mimir} supports the generation of general
instruction-tuning datasets from the same input. This dual capability ensures
that language agents developed through the platform possess both specific agent
abilities and general competencies. \textsc{Mimir} integrates these features
into a cohesive end-to-end platform, facilitating everything from the uploading
of personalized files to one-click agent fine-tuning.

Mimir 是一个流线型平台，可以提供可定制的流程，从私有知识和公开、合法的数据集中为个性化代理调优，支持从同一输入产生面向一般指导的数据集，并通过一键代理微调，以提高 GPT-4 的效率。

MIMIR：领域专家个性化代理优化的简化平台

MIMIR: A Streamlined Platform for Personalized Agent Tuning in Domain  Expertise

Vision Transformers (ViTs) achieve superior performance on various tasks
compared to convolutional neural networks (CNNs), but ViTs are also vulnerable
to adversarial attacks. Adversarial training is one of the most successful
methods to build robust CNN models. Thus, recent works explored new
methodologies for adversarial training of ViTs based on the differences between
ViTs and CNNs, such as better training strategies, preventing attention from
focusing on a single block, or discarding low-attention embeddings. However,
these methods still follow the design of traditional supervised adversarial
training, limiting the potential of adversarial training on ViTs. This paper
proposes a novel defense method, MIMIR, which aims to build a different
adversarial training methodology by utilizing Masked Image Modeling at
pre-training. We create an autoencoder that accepts adversarial examples as
input but takes the clean examples as the modeling target. Then, we create a
mutual information (MI) penalty following the idea of the Information
Bottleneck. Among the two information source inputs and corresponding
adversarial perturbation, the perturbation information is eliminated due to the
constraint of the modeling target. Next, we provide a theoretical analysis of
MIMIR using the bounds of the MI penalty. We also design two adaptive attacks
when the adversary is aware of the MIMIR defense and show that MIMIR still
performs well. The experimental results show that MIMIR improves (natural and
adversarial) accuracy on average by 4.19\% on CIFAR-10 and 5.52\% on
ImageNet-1K, compared to baselines. On Tiny-ImageNet, we obtained improved
natural accuracy of 2.99\% on average and comparable adversarial accuracy. Our
code and trained models are publicly
available\footnote{https://anonymous.4open.science/r/MIMIR-5444/README.md}.

本论文提出了一种新颖的防御方法 MIMIR，通过在预训练阶段利用 Masked Image Modeling 构建不同的对抗训练方法，从而提高 Vision Transformers 的鲁棒性和性能。实验证明，相较于基线模型，MIMIR 在 CIFAR-10 和 ImageNet-1K 上的（自然和对抗）准确率平均提高了 4.19％和 5.52％。