Focus in Explainable AI is shifting from explanations defined in terms of
low-level elements, such as input features, to explanations encoded in terms of
interpretable concepts learned from data. How to reliably acquire such concepts
is, however, still fundamentally unclear. An agreed-upon notion of concept
interpretability is missing, with the result that concepts used by both
post-hoc explainers and concept-based neural networks are acquired through a
variety of mutually incompatible strategies. Critically, most of these neglect
the human side of the problem: a representation is understandable only insofar
as it can be understood by the human at the receiving end. The key challenge in
Human-interpretable Representation Learning (HRL) is how to model and
operationalize this human element. In this work, we propose a mathematical
framework for acquiring interpretable representations suitable for both
post-hoc explainers and concept-based neural networks. Our formalization of HRL
builds on recent advances in causal representation learning and explicitly
models a human stakeholder as an external observer. This allows us to derive a
principled notion of alignment between the machine representation and the
vocabulary of concepts understood by the human. In doing so, we link alignment
and interpretability through a simple and intuitive name transfer game, and
clarify the relationship between alignment and a well-known property of
representations, namely disentanglment. We also show that alignment is linked
to the issue of undesirable correlations among concepts, also known as concept
leakage, and to content-style separation, all through a general
information-theoretic reformulation of these properties. Our conceptualization
aims to bridge the gap between the human and algorithmic sides of
interpretability and establish a stepping stone for new research on
human-interpretable representations.

通过数学框架提出了一种获取可解释表示的方法，旨在建立人与算法方面之间的理解性桥梁，并为人类可解释性表示的新研究奠定基础。