Bayesian approaches for training deep neural networks (BNNs) have received
significant interest and have been effectively utilized in a wide range of
applications. There have been several studies on the properties of posterior
concentrations of BNNs. However, most of these studies only demonstrate results
in BNN models with sparse or heavy-tailed priors. Surprisingly, no theoretical
results currently exist for BNNs using Gaussian priors, which are the most
commonly used one. The lack of theory arises from the absence of approximation
results of Deep Neural Networks (DNNs) that are non-sparse and have bounded
parameters. In this paper, we present a new approximation theory for non-sparse
DNNs with bounded parameters. Additionally, based on the approximation theory,
we show that BNNs with non-sparse general priors can achieve near-minimax
optimal posterior concentration rates to the true model.

使用贝叶斯方法进行深度神经网络（BNNs）训练在广泛应用中受到了极大关注，并且已被有效地应用于各种情况。然而，大多数关于对 BNNs 的后验集中性质的研究仅在具有稀疏或重尾先验的 BNN 模型中证明结果。令人惊讶的是，目前还没有关于使用最常用的高斯先验进行 BNNs 的理论结果存在。这种理论缺乏是由于没有非稀疏且具有有界参数的深度神经网络（DNNs）的近似结果。在本文中，我们提出了一个新的近似理论，用于具有有界参数的非稀疏 DNNs。此外，基于该近似理论，我们表明具有非稀疏一般先验的 BNNs 可以以接近最小最优后验集中速率接近真实模型。

具有权重通用先验的全连接贝叶斯神经网络的后验集中度

Posterior concentrations of fully-connected Bayesian neural networks  with general priors on the weights

In open set recognition, a classifier has to detect unknown classes that are
not known at training time. In order to recognize new categories, the
classifier has to project the input samples of known classes in very compact
and separated regions of the features space for discriminating samples of
unknown classes. Recently proposed Capsule Networks have shown to outperform
alternatives in many fields, particularly in image recognition, however they
have not been fully applied yet to open-set recognition. In capsule networks,
scalar neurons are replaced by capsule vectors or matrices, whose entries
represent different properties of objects. In our proposal, during training,
capsules features of the same known class are encouraged to match a pre-defined
gaussian, one for each class. To this end, we use the variational autoencoder
framework, with a set of gaussian priors as the approximation for the posterior
distribution. In this way, we are able to control the compactness of the
features of the same class around the center of the gaussians, thus controlling
the ability of the classifier in detecting samples from unknown classes. We
conducted several experiments and ablation of our model, obtaining state of the
art results on different datasets in the open set recognition and unknown
detection tasks.

提出了一种基于胶囊网络的 varational autoencoder 模型来进行开放式识别与未知检测任务，使用高斯先验控制胶囊特征的紧致性，获取了优秀的实验结果。

有条件变分胶囊网络用于开放集识别

Conditional Variational Capsule Network for Open Set Recognition

Sequence model learning algorithms typically maximize log-likelihood minus
the norm of the model (or minimize Hamming loss + norm). In cross-lingual
part-of-speech (POS) tagging, our target language training data consists of
sequences of sentences with word-by-word labels projected from translations in
$k$ languages for which we have labeled data, via word alignments. Our training
data is therefore very noisy, and if Rademacher complexity is high, learning
algorithms are prone to overfit. Norm-based regularization assumes a constant
width and zero mean prior. We instead propose to use the $k$ source language
models to estimate the parameters of a Gaussian prior for learning new POS
taggers. This leads to significantly better performance in multi-source
transfer set-ups. We also present a drop-out version that injects (empirical)
Gaussian noise during online learning. Finally, we note that using empirical
Gaussian priors leads to much lower Rademacher complexity, and is superior to
optimally weighted model interpolation.

使用 $k$ 源语言模型估计一个高斯先验的参数以进行多源迁移学习，同时注入（经验）高斯噪声的 dropout 版本，使用经验高斯先验可以显著降低 Rademacher 的复杂度并且优于最佳加权模型插值。

基于经验高斯先验的跨语言迁移学习

Empirical Gaussian priors for cross-lingual transfer learning

Approximate message passing (AMP) and its variants, developed based on loopy
belief propagation, are attractive for estimating a vector x from a noisy
version of z = Ax, which arises in many applications. For a large A with i. i.
d. elements, AMP can be characterized by the state evolution and exhibits fast
convergence. However, it has been shown that, AMP mayeasily diverge for a
generic A. In this work, we develop a new variant of AMP based on a unitary
transformation of the original model (hence the variant is called UT-AMP),
where the unitary matrix is available for any matrix A, e.g., the conjugate
transpose of the left singular matrix of A, or a normalized DFT (discrete
Fourier transform) matrix for any circulant A. We prove that, in the case of
Gaussian priors, UT-AMP always converges for any matrix A. It is observed that
UT-AMP is much more robust than the original AMP for difficult A and exhibits
fast convergence.
A special form of UT-AMP with a circulant A was used in our previous work
[13] for turbo equalization. This work extends it to a generic A, and provides
a theoretical investigation on the convergence.

本论文提出了 UT-AMP（一种基于原始模型的酉变换的 AMP 改进版），并证明在高斯先验条件下，UT-AMP 对于任何矩阵 A 都能收敛。UT-AMP 比原始 AMP 更加稳健，并表现出快速的收敛。