Training vision transformer networks on small datasets poses challenges. In
contrast, convolutional neural networks (CNNs) can achieve state-of-the-art
performance by leveraging their architectural inductive bias. In this paper, we
investigate whether this inductive bias can be reinterpreted as an
initialization bias within a vision transformer network. Our approach is
motivated by the finding that random impulse filters can achieve almost
comparable performance to learned filters in CNNs. We introduce a novel
initialization strategy for transformer networks that can achieve comparable
performance to CNNs on small datasets while preserving its architectural
flexibility.

我们研究了在小数据集上训练视觉转换网络所面临的挑战，发现卷积神经网络可以通过利用其体系结构归纳偏差实现最先进的性能。我们探索了这种归纳偏差是否可以重新解释为视觉转换网络中的初始化偏差，并提出了一种新的初始化策略，可以在小数据集上实现与卷积神经网络相当的性能，同时保留其架构的灵活性。

卷积初始化用于数据高效的视觉 Transformer 模型

Convolutional Initialization for Data-Efficient Vision Transformers

Parametrized quantum circuits initialized with random initial parameter
values are characterized by barren plateaus where the gradient becomes
exponentially small in the number of qubits. In this technical note we
theoretically motivate and empirically validate an initialization strategy
which can resolve the barren plateau problem for practical applications. The
technique involves randomly selecting some of the initial parameter values,
then choosing the remaining values so that the circuit is a sequence of shallow
blocks that each evaluates to the identity. This initialization limits the
effective depth of the circuits used to calculate the first parameter update so
that they cannot be stuck in a barren plateau at the start of training. In
turn, this makes some of the most compact ans\"atze usable in practice, which
was not possible before even for rather basic problems. We show empirically
that variational quantum eigensolvers and quantum neural networks initialized
using this strategy can be trained using a gradient based method.

文章提出了一种随机参数初始化生成浅层块的量子电路初始化策略，以应对梯度消失问题，实现了基于梯度的训练方法，解决了过去针对基本问题不能使用最紧凑的量子电路的问题。

解决参数化量子电路中贫瘠高原问题的初始化策略

An initialization strategy for addressing barren plateaus in  parametrized quantum circuits

Stochastic variational inference is an established way to carry out
approximate Bayesian inference for deep models. While there have been effective
proposals for good initializations for loss minimization in deep learning, far
less attention has been devoted to the issue of initialization of stochastic
variational inference. We address this by proposing a novel layer-wise
initialization strategy based on Bayesian linear models. The proposed method is
extensively validated on regression and classification tasks, including
Bayesian DeepNets and ConvNets, showing faster and better convergence compared
to alternatives inspired by the literature on initializations for loss
minimization.

本文提出了一种基于贝叶斯线性模型的新型逐层初始化策略，用于解决随机变分推断的初始化问题，并在回归、分类等任务中进行了广泛验证，与启发式初始化相比，能够实现更快更好的收敛。

深度模型变分贝叶斯的良好初始化

Good Initializations of Variational Bayes for Deep Models

Recently, a general method for analyzing the statistical accuracy of the EM
algorithm has been developed and applied to some simple latent variable models
[Balakrishnan et al. 2016]. In that method, the basin of attraction for valid
initialization is required to be a ball around the truth. Using Stein's Lemma,
we extend these results in the case of estimating the centers of a
two-component Gaussian mixture in $d$ dimensions. In particular, we
significantly expand the basin of attraction to be the intersection of a half
space and a ball around the origin. If the signal-to-noise ratio is at least a
constant multiple of $ \sqrt{d\log d} $, we show that a random initialization
strategy is feasible.

本研究拓展了 EM 算法在高斯混合模型中的应用，并通过随机初始化策略提高了统计精确度。