When the prediction of a black-box machine learning model deviates from the
true observation, what can be said about the reason behind that deviation? This
is a fundamental and ubiquitous question that the end user in a business or
industrial AI application often asks. The deviation may be due to a sub-optimal
black-box model, or it may be simply because the sample in question is an
outlier. In either case, one would ideally wish to obtain some form of
attribution score -- a value indicative of the extent to which an input
variable is responsible for the anomaly.
In the present paper we address this task of ``anomaly attribution,''
particularly in the setting in which the model is black-box and the training
data are not available. Specifically, we propose a novel likelihood-based
attribution framework we call the ``likelihood compensation (LC),'' in which
the responsibility score is equated with the correction on each input variable
needed to attain the highest possible likelihood. We begin by showing formally
why mainstream model-agnostic explanation methods, such as the local linear
surrogate modeling and Shapley values, are not designed to explain anomalies.
In particular, we show that they are ``deviation-agnostic,'' namely, that their
explanations are blind to the fact that there is a deviation in the model
prediction for the sample of interest. We do this by positioning these existing
methods under the unified umbrella of a function family we call the
``integrated gradient family.'' We validate the effectiveness of the proposed
LC approach using publicly available data sets. We also conduct a case study
with a real-world building energy prediction task and confirm its usefulness in
practice based on expert feedback.

介绍了一种基于可能性补偿的异常归属框架，用于黑匣子机器学习模型下对异常进行归属。通过对比现有的模型无关解释方法的局限性，利用独立梯度家族函数来辨别异常，证明了该方法的实用性和有效性。