A classifier is, in its essence, a function which takes an input and returns
the class of the input and implicitly assumes an underlying distribution. We
argue in this article that one has to move away from this basic tenet to obtain
generalisation across distributions. Specifically, the class of the sample
should depend on the points from its context distribution for better
generalisation across distributions.
How does one achieve this? The key idea is to adapt the outputs of each
neuron of the network to its context distribution. We propose quantile
activation, QACT, which, in simple terms, outputs the relative quantile of the
sample in its context distribution, instead of the actual values in traditional
networks.
The scope of this article is to validate the proposed activation across
several experimental settings, and compare it with conventional techniques. For
this, we use the datasets developed to test robustness against distortions
CIFAR10C, CIFAR100C, MNISTC, TinyImagenetC, and show that we achieve a
significantly higher generalisation across distortions than the conventional
classifiers, across different architectures. Although this paper is only a
proof of concept, we surprisingly find that this approach outperforms
DINOv2(small) at large distortions, even though DINOv2 is trained with a far
bigger network on a considerably larger dataset.

通过将每个神经元的输出调整为其上下文分布，本文提出了量化激活（QACT）方法，该方法在简单网络中输出样本在其上下文分布中的相对分位数，以实现跨分布的更好泛化。我们验证了该方法在几个实验设置上的有效性，并将其与传统技术进行比较，结果表明我们在对抗扭曲的不同架构下实现了显著更高的泛化性能。尽管本文只是一个概念验证，但令人惊讶的是，即使 DINOv2 使用更大的网络和更大的数据集进行训练，在大的扭曲下该方法依然胜过 DINOv2（small）模型。