The fight between discriminative versus generative goes deep, in both the
study of artificial and natural intelligence. In our view, both camps have
complementary value, so, we sought to synergistic combine them. Here, we
propose a methodology to convert deep discriminative networks to kernel
generative networks. We leveraged the fact that deep models, including both
random forests and deep networks, learn internal representations which are
unions of polytopes with affine activation functions to conceptualize them both
as generalized partitioning rules. From that perspective, we used foundational
results on the relationship between histogram rules and kernel density
estimators to obtain class conditional kernel density estimators from the deep
models. We then studied the trade-offs we observed from implementing this
strategy in low-dimensional settings, both theoretically and empirically, as a
first step towards understanding. Theoretically, we show conditions under which
our generative models are more efficient than the corresponding discriminative
approaches. Empirically, when sample sizes are relatively high, the
discriminative models tend to perform as well or better on discriminative
metrics, such as classification rates and posterior calibration. However, when
sample sizes are relatively low, the generative models outperform the
discriminative ones even on discriminative metrics. Moreover, the generative
ones can also sample from the distribution, obtain smoother posteriors, and
extrapolate beyond the convex hull of the training data to handle OOD inputs
more reasonably. Via human experiments we illustrate that our kernel generative
networks (Kragen) behave more like humans than deep discriminative networks. We
believe this approach may be an important step in unifying the thinking and
approaches across the discriminative and generative divide.

该论文提出了一种 kernel generative networks 方法，将深度鉴别网络转换为核生成网络，实现了鉴别模型和生成模型的协同配合，探讨了在低维环境下两种模型的应用时的理论和经验表现，证明了它比对应的鉴别学习方法更有效，并可以在样本量较少时实现更优的表现，不仅可以获得更平滑的后验概率，而且可以超出训练数据的凸包范围，更好地处理 OOD 输入。

深度判别到核生成建模

Deep discriminative to kernel generative modeling

We propose one-class support measure machines (OCSMMs) for group anomaly
detection which aims at recognizing anomalous aggregate behaviors of data
points. The OCSMMs generalize well-known one-class support vector machines
(OCSVMs) to a space of probability measures. By formulating the problem as
quantile estimation on distributions, we can establish an interesting
connection to the OCSVMs and variable kernel density estimators (VKDEs) over
the input space on which the distributions are defined, bridging the gap
between large-margin methods and kernel density estimators. In particular, we
show that various types of VKDEs can be considered as solutions to a class of
regularization problems studied in this paper. Experiments on Sloan Digital Sky
Survey dataset and High Energy Particle Physics dataset demonstrate the
benefits of the proposed framework in real-world applications.

该论文提出了一种名为 OCSMMs 的分类器，可以用于识别数据点的异常聚合行为，该分类器将正常行为视为极端少数事件，并生成一个概率分布进行判定。论文连接了大边界方法和核密度估计，提出以分布量化估计的方式解决问题。在 Sloan 数码天景调查数据集和高能物理学数据集上开展的实验表明了该框架在实际应用中的好处。