Within the realm of deep learning, the interpretability of Convolutional
Neural Networks (CNNs), particularly in the context of image classification
tasks, remains a formidable challenge. To this end we present a neurosymbolic
framework, NeSyFOLD-G that generates a symbolic rule-set using the last layer
kernels of the CNN to make its underlying knowledge interpretable. What makes
NeSyFOLD-G different from other similar frameworks is that we first find groups
of similar kernels in the CNN (kernel-grouping) using the cosine-similarity
between the feature maps generated by various kernels. Once such kernel groups
are found, we binarize each kernel group's output in the CNN and use it to
generate a binarization table which serves as input data to FOLD-SE-M which is
a Rule Based Machine Learning (RBML) algorithm. FOLD-SE-M then generates a
rule-set that can be used to make predictions. We present a novel kernel
grouping algorithm and show that grouping similar kernels leads to a
significant reduction in the size of the rule-set generated by FOLD-SE-M,
consequently, improving the interpretability. This rule-set symbolically
encapsulates the connectionist knowledge of the trained CNN. The rule-set can
be viewed as a normal logic program wherein each predicate's truth value
depends on a kernel group in the CNN. Each predicate in the rule-set is mapped
to a concept using a few semantic segmentation masks of the images used for
training, to make it human-understandable. The last layers of the CNN can then
be replaced by this rule-set to obtain the NeSy-G model which can then be used
for the image classification task. The goal directed ASP system s(CASP) can be
used to obtain the justification of any prediction made using the NeSy-G model.
We also propose a novel algorithm for labeling each predicate in the rule-set
with the semantic concept(s) that its corresponding kernel group represents.

通过提出 NeSyFOLD-G 神经符号化框架，本研究使用卷积神经网络的最后一层核心生成符号规则集，提高了深度学习模型解释性，并通过核心分组算法减少规则集大小。

利用逻辑编程和核分组提高卷积神经网络的可解释性

Using Logic Programming and Kernel-Grouping for Improving  Interpretability of Convolutional Neural Networks

Binary Neural Network (BNN) represents convolution weights with 1-bit values,
which enhances the efficiency of storage and computation. This paper is
motivated by a previously revealed phenomenon that the binary kernels in
successful BNNs are nearly power-law distributed: their values are mostly
clustered into a small number of codewords. This phenomenon encourages us to
compact typical BNNs and obtain further close performance through learning
non-repetitive kernels within a binary kernel subspace. Specifically, we regard
the binarization process as kernel grouping in terms of a binary codebook, and
our task lies in learning to select a smaller subset of codewords from the full
codebook. We then leverage the Gumbel-Sinkhorn technique to approximate the
codeword selection process, and develop the Permutation Straight-Through
Estimator (PSTE) that is able to not only optimize the selection process
end-to-end but also maintain the non-repetitive occupancy of selected
codewords. Experiments verify that our method reduces both the model size and
bit-wise computational costs, and achieves accuracy improvements compared with
state-of-the-art BNNs under comparable budgets.

本研究提出了一种通过学习二元核子空间内的非重复核来压缩典型的二进制神经网络模型的方法，通过 Gumbel-Sinkhorn 技术近似码字选择过程，在可比较的预算下实现了模型大小和计算成本的降低，并获得了与现有技术相比的精度提升。