Society's capacity for algorithmic problem-solving has never been greater.
Artificial Intelligence is now applied across more domains than ever, a
consequence of powerful abstractions, abundant data, and accessible software.
As capabilities have expanded, so have risks, with models often deployed
without fully understanding their potential impacts. Interpretable and
interactive machine learning aims to make complex models more transparent and
controllable, enhancing user agency. This review synthesizes key principles
from the growing literature in this field.
We first introduce precise vocabulary for discussing interpretability, like
the distinction between glass box and explainable algorithms. We then explore
connections to classical statistical and design principles, like parsimony and
the gulfs of interaction. Basic explainability techniques -- including learned
embeddings, integrated gradients, and concept bottlenecks -- are illustrated
with a simple case study. We also review criteria for objectively evaluating
interpretability approaches. Throughout, we underscore the importance of
considering audience goals when designing interactive algorithmic systems.
Finally, we outline open challenges and discuss the potential role of data
science in addressing them. Code to reproduce all examples can be found at
this https URL

社会对算法问题解决的能力前所未有地增强。人工智能如今应用于更多的领域，这是强大的抽象化、丰富的数据和易于使用的软件的结果。随着能力的扩展，风险也增加了，因为模型常常在没有完全理解其潜在影响的情况下部署。可解释且交互式的机器学习旨在使复杂模型更加透明和可控，增强用户机构。本综述从这一领域的不断增长的文献中综合了关键原则。

可解释的人工智能的数据科学原理

Data Science Principles for Interpretable and Explainable AI

Programming often involves converting detailed and complex specifications
into code, a process during which developers typically utilize visual aids to
more effectively convey concepts. While recent developments in Large Multimodal
Models have demonstrated remarkable abilities in visual reasoning and
mathematical tasks, there is little work on investigating whether these models
can effectively interpret visual elements for code generation. To this end, we
present MMCode, the first multi-modal coding dataset for evaluating algorithmic
problem-solving skills in visually rich contexts. MMCode contains 3,548
questions and 6,620 images collected from real-world programming challenges
harvested from 10 code competition websites, presenting significant challenges
due to the extreme demand for reasoning abilities. Our experiment results show
that current state-of-the-art models struggle to solve these problems. The
results highlight the lack of powerful vision-code models, and we hope MMCode
can serve as an inspiration for future works in this domain. The data and code
are publicly available at this https URL

在代码编写过程中，开发人员通常利用视觉辅助工具更有效地传达概念。然而，最近在大型多模态模型方面的发展表明在视觉推理和数学任务方面取得了显著能力，但在研究这些模型是否能够有效地解释用于代码生成的视觉元素方面几乎没有相关工作。为此，我们提出了 MMCode，这是第一个用于在视觉丰富的环境中评估算法问题解决能力的多模态编码数据集，包含了 3548 个问题和 6620 张图片，这些数据来自于 10 个代码竞赛网站采集的现实世界的编程挑战，由于对推理能力的极高要求而具有显著的挑战性。我们的实验结果表明，现有最先进的模型在解决这些问题时存在困难。这些结果突显了缺乏强大的视觉 - 代码模型，我们希望 MMCode 可以成为这个领域未来工作的启示。该数据集和代码可以在提供的网址公开获取。

使用丰富视觉编程问题评估多模态编程大型语言模型

MMCode: Evaluating Multi-Modal Code Large Language Models with Visually  Rich Programming Problems

Recent research in mechanistic interpretability has attempted to
reverse-engineer Transformer models by carefully inspecting network weights and
activations. However, these approaches require considerable manual effort and
still fall short of providing complete, faithful descriptions of the underlying
algorithms. In this work, we introduce a procedure for training Transformers
that are mechanistically interpretable by design. We build on RASP [Weiss et
al., 2021], a programming language that can be compiled into Transformer
weights. Instead of compiling human-written programs into Transformers, we
design a modified Transformer that can be trained using gradient-based
optimization and then be automatically converted into a discrete,
human-readable program. We refer to these models as Transformer Programs. To
validate our approach, we learn Transformer Programs for a variety of problems,
including an in-context learning task, a suite of algorithmic problems (e.g.
sorting, recognizing Dyck-languages), and NLP tasks including named entity
recognition and text classification. The Transformer Programs can automatically
find reasonable solutions, performing on par with standard Transformers of
comparable size; and, more importantly, they are easy to interpret. To
demonstrate these advantages, we convert Transformers into Python programs and
use off-the-shelf code analysis tools to debug model errors and identify the
``circuits'' used to solve different sub-problems. We hope that Transformer
Programs open a new path toward the goal of intrinsically interpretable machine
learning.

本文介绍的一种采用编程语言设计的可调试 Transformer 程序可以替代机器学习传统编写的 Transformers 来解决一系列算法问题和 NLP 任务。