Covariance and Hessian matrices have been analyzed separately in the
literature for classification problems. However, integrating these matrices has
the potential to enhance their combined power in improving classification
performance. We present a novel approach that combines the eigenanalysis of a
covariance matrix evaluated on a training set with a Hessian matrix evaluated
on a deep learning model to achieve optimal class separability in binary
classification tasks. Our approach is substantiated by formal proofs that
establish its capability to maximize between-class mean distance and minimize
within-class variances. By projecting data into the combined space of the most
relevant eigendirections from both matrices, we achieve optimal class
separability as per the linear discriminant analysis (LDA) criteria. Empirical
validation across neural and health datasets consistently supports our
theoretical framework and demonstrates that our method outperforms established
methods. Our method stands out by addressing both LDA criteria, unlike PCA and
the Hessian method, which predominantly emphasize one criterion each. This
comprehensive approach captures intricate patterns and relationships, enhancing
classification performance. Furthermore, through the utilization of both LDA
criteria, our method outperforms LDA itself by leveraging higher-dimensional
feature spaces, in accordance with Cover's theorem, which favors linear
separability in higher dimensions. Our method also surpasses kernel-based
methods and manifold learning techniques in performance. Additionally, our
approach sheds light on complex DNN decision-making, rendering them
comprehensible within a 2D space.

一种新方法将训练集上评估的协方差矩阵的特征分解与在深度学习模型上评估的 Hessian 矩阵相结合，实现了二分类任务中的最优类别可分性，通过投影数据到两个矩阵的最相关特征方向组合空间，优化类别可分性，实验证明我们的方法胜过传统方法，同时突出处理线性判别分析标准。