Explainability of neural network prediction is essential to understand
feature importance and gain interpretable insight into neural network
performance. In this work, model explanations are fed back to the feed-forward
training to help the model generalize better. To this extent, a custom weighted
loss where the weights are generated by considering the Euclidean distances
between true LIME (Local Interpretable Model-Agnostic Explanations)
explanations and model-predicted LIME explanations is proposed. Also, in
practical training scenarios, developing a solution that can help the model
learn sequentially without losing information on previous data distribution is
imperative due to the unavailability of all the training data at once. Thus,
the framework known as XAI-Increment incorporates the custom weighted loss
developed with elastic weight consolidation (EWC), to maintain performance in
sequential testing sets. Finally, the training procedure involving the custom
weighted loss shows around 1% accuracy improvement compared to the traditional
loss based training for the keyword spotting task on the Google Speech Commands
dataset and also shows low loss of information when coupled with EWC in the
incremental learning setup.

本文提出了一种自定义加权损失函数，其中考虑了真实 LIME 解释和模型预测 LIME 解释之间的欧几里得距离，用于提高模型的泛化能力，并通过增量学习设置来验证其有效性。与传统损失函数相比，使用自定义加权损失函数的训练程序对于 Google 语音命令数据集上的关键字识别任务表现出 1％的准确率提高，并在与 EWC 相结合的增量学习设置中表现出低信息损失。

XAI-Increment: 一种利用 LIME 解释来改进增量学习的新方法

XAI-Increment: A Novel Approach Leveraging LIME Explanations for Improved Incremental Learning

Intuitively, one would expect accuracy of a trained neural network's
prediction on test samples to correlate with how densely the samples are
surrounded by seen training samples in representation space. We find that a
bound on empirical training error smoothed across linear activation regions
scales inversely with training sample density in representation space.
Empirically, we verify this bound is a strong predictor of the inaccuracy of
the network's prediction on test samples. For unseen test sets, including those
with out-of-distribution samples, ranking test samples by their local region's
error bound and discarding samples with the highest bounds raises prediction
accuracy by up to 20% in absolute terms for image classification datasets, on
average over thresholds.

通过研究表明，在表示空间中，训练样本密度的倒数平滑处理后的经验训练误差边界与测试样本的预测不准确性密切相关，通过在测试集中排列样本边界值并放弃最高边界值的样本，平均阈值下对图像分类数据集的预测准确性提高了 20％。