Intelligence relies on an agent's knowledge of what it does not know. This
capability can be assessed based on the quality of joint predictions of labels
across multiple inputs. In principle, ensemble-based approaches produce
effective joint predictions, but the computational costs of training large
ensembles can become prohibitive. We introduce the epinet: an architecture that
can supplement any conventional neural network, including large pretrained
models, and can be trained with modest incremental computation to estimate
uncertainty. With an epinet, conventional neural networks outperform very large
ensembles, consisting of hundreds or more particles, with orders of magnitude
less computation. The epinet does not fit the traditional framework of Bayesian
neural networks. To accommodate development of approaches beyond BNNs, such as
the epinet, we introduce the epistemic neural network (ENN) as an interface for
models that produce joint predictions.

本文介绍了一种能够辅助任何传统神经网络，包括大型预训练模型，能够用较少的计算量来预测不确定性的架构 ——epinet。使用 epinet，传统神经网络在预测标签上的表现优于包含数百个或更多粒子的非常大的许多集合，而且使用的计算资源和数据也大大降低了。

认知神经网络

Epistemic Neural Networks

An intriguing property of deep neural networks is the existence of
adversarial examples, which can transfer among different architectures. These
transferable adversarial examples may severely hinder deep neural network-based
applications. Previous works mostly study the transferability using small scale
datasets. In this work, we are the first to conduct an extensive study of the
transferability over large models and a large scale dataset, and we are also
the first to study the transferability of targeted adversarial examples with
their target labels. We study both non-targeted and targeted adversarial
examples, and show that while transferable non-targeted adversarial examples
are easy to find, targeted adversarial examples generated using existing
approaches almost never transfer with their target labels. Therefore, we
propose novel ensemble-based approaches to generating transferable adversarial
examples. Using such approaches, we observe a large proportion of targeted
adversarial examples that are able to transfer with their target labels for the
first time. We also present some geometric studies to help understanding the
transferable adversarial examples. Finally, we show that the adversarial
examples generated using ensemble-based approaches can successfully attack
Clarifai.com, which is a black-box image classification system.

本文首次对大规模数据集和大型模型进行了对抗样本的可迁移性的研究，同时也是首次研究了设计有目标的对抗样本在其目标标签之间的可转移性。通过新颖的集成方法，本文发现大量有目标的对抗样本可以成功地与其目标标签一起转移，并且这些使用集成方法生成的对抗样本可以成功地攻击黑匣子图像分类系统 Clarifai.com。