Neural network have achieved remarkable successes in many scientific fields.
However, the interpretability of the neural network model is still a major
bottlenecks to deploy such technique into our daily life. The challenge can
dive into the non-linear behavior of the neural network, which rises a critical
question that how a model use input feature to make a decision. The classical
approach to address this challenge is feature attribution, which assigns an
important score to each input feature and reveal its importance of current
prediction. However, current feature attribution approaches often indicate the
importance of each input feature without detail of how they are actually
processed by a model internally. These attribution approaches often raise a
concern that whether they highlight correct features for a model prediction.
For a neural network model, the non-linear behavior is often caused by
non-linear activation units of a model. However, the computation behavior of a
prediction from a neural network model is locally linear, because one
prediction has only one activation pattern. Base on the observation, we propose
an instance-wise linearization approach to reformulates the forward computation
process of a neural network prediction. This approach reformulates different
layers of convolution neural networks into linear matrix multiplication.
Aggregating all layers' computation, a prediction complex convolution neural
network operations can be described as a linear matrix multiplication $F(x) = W
\cdot x + b$. This equation can not only provides a feature attribution map
that highlights the important of the input features but also tells how each
input feature contributes to a prediction exactly. Furthermore, we discuss the
application of this technique in both supervise classification and unsupervised
neural network learning parametric t-SNE dimension reduction.

神经网络在科学领域取得了显著的成功，但是其模型的可解释性仍然是将这种技术应用于我们日常生活的主要瓶颈之一。本文提出了一种基于实例的线性化方法，来重新定义神经网络预测的前向计算过程，从而提供了一个突出输入特征重要性并准确解释每个输入特征对预测的贡献的特征归因图。此技术在有监督分类和无监督神经网络学习参数化 t-SNE 降维中的应用也进行了讨论。