We derive closed-form expressions for the Bayes optimal decision boundaries
in binary classification of high dimensional overlapping Gaussian mixture model
(GMM) data, and show how they depend on the eigenstructure of the class
covariances, for particularly interesting structured data. We empirically
demonstrate, through experiments on synthetic GMMs inspired by real-world data,
that deep neural networks trained for classification, learn predictors which
approximate the derived optimal classifiers. We further extend our study to
networks trained on authentic data, observing that decision thresholds
correlate with the covariance eigenvectors rather than the eigenvalues,
mirroring our GMM analysis. This provides theoretical insights regarding neural
networks' ability to perform probabilistic inference and distill statistical
patterns from intricate distributions.

高维重叠高斯混合模型 (GMM) 数据的二元分类的贝叶斯最优决策边界的闭合表达式会根据类协方差的特征结构而变化。通过对由真实数据启发的合成 GMM 数据进行的实验，我们从经验上证明了深度神经网络在分类训练中学习预测器可以近似得到最优分类器。我们进一步拓展了对真实数据训练的网络的研究，观察到决策阈值与协方差的特征向量相关而不是特征值，从而反映了我们 GMM 分析的结果。这为神经网络从错综复杂的分布中执行概率推断和提取统计模式的能力提供了理论洞见。