Understanding which concepts models can and cannot represent has been
fundamental to many tasks: from effective and responsible use of models to
detecting out of distribution data. We introduce Gaussian process probes (GPP),
a unified and simple framework for probing and measuring uncertainty about
concepts represented by models. As a Bayesian extension of linear probing
methods, GPP asks what kind of distribution over classifiers (of concepts) is
induced by the model. This distribution can be used to measure both what the
model represents and how confident the probe is about what the model
represents. GPP can be applied to any pre-trained model with vector
representations of inputs (e.g., activations). It does not require access to
training data, gradients, or the architecture. We validate GPP on datasets
containing both synthetic and real images. Our experiments show it can (1)
probe a model's representations of concepts even with a very small number of
examples, (2) accurately measure both epistemic uncertainty (how confident the
probe is) and aleatory uncertainty (how fuzzy the concepts are to the model),
and (3) detect out of distribution data using those uncertainty measures as
well as classic methods do. By using Gaussian processes to expand what probing
can offer, GPP provides a data-efficient, versatile and uncertainty-aware tool
for understanding and evaluating the capabilities of machine learning models.

引入了高斯过程探针 (Gaussian process probes, GPP) 框架，是一种确定模型表示哪些概念的方法，并测量其不确定性，从而为理解和评估机器学习模型的能力提供了一种数据效率高、多功能和具有不确定性意识的工具。

高斯过程探针（GPP）用于带不确定性的探测

Gaussian Process Probes (GPP) for Uncertainty-Aware Probing

We propose a method for training a deterministic deep model that can find and
reject out of distribution data points at test time with a single forward pass.
Our approach, deterministic uncertainty quantification (DUQ), builds upon ideas
of RBF networks. We scale training in these with a novel loss function and
centroid updating scheme and match the accuracy of softmax models. By enforcing
detectability of changes in the input using a gradient penalty, we are able to
reliably detect out of distribution data. Our uncertainty quantification scales
well to large datasets, and using a single model, we improve upon or match Deep
Ensembles in out of distribution detection on notable difficult dataset pairs
such as FashionMNIST vs. MNIST, and CIFAR-10 vs. SVHN.

论文提出了一种通过单次前向传递来训练确定性深度模型的方法，该深度模型可以找到并拒绝分布外的数据点。此外，论文还提出了一种基于确定性不确定性量化（DUQ）的建议，并描述了一种新的损失函数和质心更新策略。通过引入梯度惩罚，我们能够可靠地检测分布外的数据，并且可以在大型数据集上缩放不确定性量化。最后，作者在一些难以检测的数据集上，如 FashionMNIST vs. MNIST 和 CIFAR-10 vs. SVHN，使用单个模型得出了比 Deep Ensembles 更好的结果。