This paper describes MAIA, a Multimodal Automated Interpretability Agent.
MAIA is a system that uses neural models to automate neural model understanding
tasks like feature interpretation and failure mode discovery. It equips a
pre-trained vision-language model with a set of tools that support iterative
experimentation on subcomponents of other models to explain their behavior.
These include tools commonly used by human interpretability researchers: for
synthesizing and editing inputs, computing maximally activating exemplars from
real-world datasets, and summarizing and describing experimental results.
Interpretability experiments proposed by MAIA compose these tools to describe
and explain system behavior. We evaluate applications of MAIA to computer
vision models. We first characterize MAIA's ability to describe (neuron-level)
features in learned representations of images. Across several trained models
and a novel dataset of synthetic vision neurons with paired ground-truth
descriptions, MAIA produces descriptions comparable to those generated by
expert human experimenters. We then show that MAIA can aid in two additional
interpretability tasks: reducing sensitivity to spurious features, and
automatically identifying inputs likely to be mis-classified.

这篇论文介绍了 MAIA，一种多模态自动解释性代理。MAIA 是一个使用神经模型自动化神经模型理解任务的系统，如特征解释和故障模式发现。它通过提供一系列工具来对其他模型的子组件进行迭代实验，从而对其行为进行解释。这些工具包括人工解释性研究人员常用的工具：用于合成和编辑输入、计算最大激活样本、以及总结和描述实验结果。MAIA 提出的解释性实验将这些工具组合起来描述和解释系统行为。我们评估了 MAIA 在计算机视觉模型中的应用。首先，我们描述了 MAIA 在学习到的图像表示的特征（神经元级别）中的描述能力。在多个经过训练的模型和一个包含真实描述的人工生成视觉神经元数据集中，MAIA 生成了与由专家人工实验者生成的描述相当的结果。然后，我们展示了 MAIA 在两个附加的可解释性任务中的应用：降低对虚假特征的敏感性和自动识别可能被错误分类的输入。