An explanation of a machine learning model is considered "faithful" if it
accurately reflects the model's decision-making process. However, explanations
such as feature attributions for deep learning are not guaranteed to be
faithful, and can produce potentially misleading interpretations. In this work,
we develop Sum-of-Parts (SOP), a class of models whose predictions come with
grouped feature attributions that are faithful-by-construction. This model
decomposes a prediction into an interpretable sum of scores, each of which is
directly attributable to a sparse group of features. We evaluate SOP on
benchmarks with standard interpretability metrics, and in a case study, we use
the faithful explanations from SOP to help astrophysicists discover new
knowledge about galaxy formation.

通过解释机器学习模型的决策过程来提供准确反映的解释被认为是 “忠实的”。本研究开发了一种名为 SOP 的模型类，其预测结果附带了忠实的分组特征归因。该模型将预测分解为可解释的得分总和，其中每个得分直接归因于一组稀疏特征。我们通过标准解释性度量和一个案例研究对 SOP 进行评估，并使用 SOP 提供的忠实解释帮助天体物理学家发现有关星系形成的新知识。