Pruning deep neural networks is a widely used strategy to alleviate the
computational burden in machine learning. Overwhelming empirical evidence
suggests that pruned models retain very high accuracy even with a tiny fraction
of parameters. However, relatively little work has gone into characterising the
small pruned networks obtained, beyond a measure of their accuracy. In this
paper, we use the sparse double descent approach to identify univocally and
characterise pruned models associated with classification tasks. We observe
empirically that, for a given task, iterative magnitude pruning (IMP) tends to
converge to networks of comparable sizes even when starting from full networks
with sizes ranging over orders of magnitude. We analyse the best pruned models
in a controlled experimental setup and show that their number of parameters
reflects task difficulty and that they are much better than full networks at
capturing the true conditional probability distribution of the labels. On real
data, we similarly observe that pruned models are less prone to overconfident
predictions. Our results suggest that pruned models obtained via IMP not only
have advantageous computational properties but also provide a better
representation of uncertainty in learning.

本文介绍了一种使用稀疏双下降方法鉴定和表征与分类任务相关的剪枝模型，该方法对网络大小变化具有鲁棒性，并表明剪枝模型不仅具有更好的计算性能，而且可以更好地表示学习中的不确定性。

标签噪声下量化彩票：准确性，校准和复杂度

Quantifying lottery tickets under label noise: accuracy, calibration,  and complexity

Generative Flow Networks (GFlowNets), a class of generative models over
discrete and structured sample spaces, have been previously applied to the
problem of inferring the marginal posterior distribution over the directed
acyclic graph (DAG) of a Bayesian Network, given a dataset of observations.
Based on recent advances extending this framework to non-discrete sample
spaces, we propose in this paper to approximate the joint posterior over not
only the structure of a Bayesian Network, but also the parameters of its
conditional probability distributions. We use a single GFlowNet whose sampling
policy follows a two-phase process: the DAG is first generated sequentially one
edge at a time, and then the corresponding parameters are picked once the full
structure is known. Since the parameters are included in the posterior
distribution, this leaves more flexibility for the local probability models of
the Bayesian Network, making our approach applicable even to non-linear models
parametrized by neural networks. We show that our method, called JSP-GFN,
offers an accurate approximation of the joint posterior, while comparing
favorably against existing methods on both simulated and real data.

该研究论文提出了一种名为 JSP-GFN 的新方法，该方法使用一个生成流网络对贝叶斯网络结构和条件概率分布的参数进行共同后验分布近似，以适用于非线性模型，并在模拟和真实数据上进行验证。

使用单一生成流网络进行图形结构与参数的联合贝叶斯推断

Joint Bayesian Inference of Graphical Structure and Parameters with a  Single Generative Flow Network

We consider the problem of estimating the conditional probability
distribution of missing values given the observed ones. We propose an approach,
which combines the flexibility of deep neural networks with the simplicity of
Gaussian mixture models (GMMs). Given an incomplete data point, our neural
network returns the parameters of Gaussian distribution (in the form of Factor
Analyzers model) representing the corresponding conditional density. We
experimentally verify that our model provides better log-likelihood than
conditional GMM trained in a typical way. Moreover, imputation obtained by
replacing missing values using the mean vector of our model looks visually
plausible.

本研究提出了一种结合深度神经网络灵活性和高斯混合模型简洁性的方法来估计缺失值的条件概率分布，并实验验证了我们的模型在插补缺失值方面的有效性。