We consider generating explanations for neural networks in cases where the
network's training data is not accessible, for instance due to privacy or
safety issues. Recently, $\mathcal{I}$-Nets have been proposed as a sample-free
approach to post-hoc, global model interpretability that does not require
access to training data. They formulate interpretation as a machine learning
task that maps network representations (parameters) to a representation of an
interpretable function. In this paper, we extend the $\mathcal{I}$-Net
framework to the cases of standard and soft decision trees as surrogate models.
We propose a suitable decision tree representation and design of the
corresponding $\mathcal{I}$-Net output layers. Furthermore, we make
$\mathcal{I}$-Nets applicable to real-world tasks by considering more realistic
distributions when generating the $\mathcal{I}$-Net's training data. We
empirically evaluate our approach against traditional global, post-hoc
interpretability approaches and show that it achieves superior results when the
training data is not accessible.

本文介绍了一种机器学习任务来理解神经网络中的可解释函数，并将该框架扩展到标准和软决策树作为替代模型的情况，并通过考虑更现实的分布来生成训练数据来使其适用于更加现实的任务，并在实验中表明其具有优秀的性能。

无需训练数据，解释神经网络

Explaining Neural Networks without Access to Training Data

Ensembles of decision trees are a useful tool for obtaining for obtaining
flexible estimates of regression functions. Examples of these methods include
gradient boosted decision trees, random forests, and Bayesian CART. Two
potential shortcomings of tree ensembles are their lack of smoothness and
vulnerability to the curse of dimensionality. We show that these issues can be
overcome by instead considering sparsity inducing soft decision trees in which
the decisions are treated as probabilistic. We implement this in the context of
the Bayesian additive regression trees framework, and illustrate its promising
performance through testing on benchmark datasets. We provide strong
theoretical support for our methodology by showing that the posterior
distribution concentrates at the minimax rate (up-to a logarithmic factor) for
sparse functions and functions with additive structures in the high-dimensional
regime where the dimensionality of the covariate space is allowed to grow near
exponentially in the sample size. Our method also adapts to the unknown
smoothness and sparsity levels, and can be implemented by making minimal
modifications to existing BART algorithms.

使用稀疏感应的 soft decision trees 取代传统的决策树集成方法，可以克服它们的光滑性差和受维度灾难的困扰。在 Bayesian additive regression trees 框架下实现该方法，理论上得到强大的支持，并在基准数据集上表现出良好的性能。