Deep neural networks (NNs) are known to lack uncertainty estimates and
struggle to incorporate new data. We present a method that mitigates these
issues by converting NNs from weight space to function space, via a dual
parameterization. Importantly, the dual parameterization enables us to
formulate a sparse representation that captures information from the entire
data set. This offers a compact and principled way of capturing uncertainty and
enables us to incorporate new data without retraining whilst retaining
predictive performance. We provide proof-of-concept demonstrations with the
proposed approach for quantifying uncertainty in supervised learning on UCI
benchmark tasks.

我们提出了一种方法，通过将神经网络从权重空间转换为函数空间，来解决深度神经网络缺乏不确定性估计和难以整合新数据的问题。这种双重参数化的方法使我们能够制定出捕捉整个数据集信息的稀疏表示，为捕捉不确定性提供了一种紧凑而有原则的方式，并且能够在保持预测性能的同时整合新数据而无需重新训练。我们通过在 UCI 基准任务上的有关监督学习的不确定性量化的概念验证演示来支持所提出的方法。

神经网络的稀疏函数空间表示

Sparse Function-space Representation of Neural Networks

What can pre-trained multilingual sequence-to-sequence models like mBART
contribute to translating low-resource languages? We conduct a thorough
empirical experiment in 10 languages to ascertain this, considering five
factors: (1) the amount of fine-tuning data, (2) the noise in the fine-tuning
data, (3) the amount of pre-training data in the model, (4) the impact of
domain mismatch, and (5) language typology. In addition to yielding several
heuristics, the experiments form a framework for evaluating the data
sensitivities of machine translation systems. While mBART is robust to domain
differences, its translations for unseen and typologically distant languages
remain below 3.0 BLEU. In answer to our title's question, mBART is not a
low-resource panacea; we therefore encourage shifting the emphasis from new
models to new data.

通过在 10 种语言上进行完备实验，考虑了五个因素：（1）微调数据量，（2）微调数据中的噪声，（3）模型预训练数据量，（4）领域不匹配的影响以及（5）语言类型学，我们试图确定预训练多语言序列到序列模型（如 mBART）对翻译低资源语言有何贡献，并提出了许多启发式方法。实验结果显示，mBART 对领域差异具有鲁棒性，但对未知和语言类型学差异较大的语言的翻译仍低于 3.0 BLEU。因此，我们鼓励将重点从新模型转向新数据。