The infinite-width limit of neural networks (NNs) has garnered significant
attention as a theoretical framework for analyzing the behavior of large-scale,
overparametrized networks. By approaching infinite width, NNs effectively
converge to a linear model with features characterized by the neural tangent
kernel (NTK). This establishes a connection between NNs and kernel methods, the
latter of which are well understood. Based on this link, theoretical benefits
and algorithmic improvements have been hypothesized and empirically
demonstrated in synthetic architectures. These advantages include faster
optimization, reliable uncertainty quantification and improved continual
learning. However, current results quantifying the rate of convergence to the
kernel regime suggest that exploiting these benefits requires architectures
that are orders of magnitude wider than they are deep. This assumption raises
concerns that practically relevant architectures do not exhibit behavior as
predicted via the NTK. In this work, we empirically investigate whether the
limiting regime either describes the behavior of large-width architectures used
in practice or is informative for algorithmic improvements. Our empirical
results demonstrate that this is not the case in optimization, uncertainty
quantification or continual learning. This observed disconnect between theory
and practice calls into question the practical relevance of the infinite-width
limit.

通过对无限宽度极限进行实证研究，我们的结果表明：无限宽度极限理论不适用于实际的优化、不确定性量化和持续学习问题，进而对于无限宽度极限的实际相关性提出了质疑。

关于超参数神经网络理论与实践之间的脱节

On the Disconnect Between Theory and Practice of Overparametrized Neural  Networks

Fairness in AI has garnered quite some attention in research, and
increasingly also in society. The so-called "Impossibility Theorem" has been
one of the more striking research results with both theoretical and practical
consequences, as it states that satisfying a certain combination of fairness
measures is impossible. To date, this negative result has not yet been
complemented with a positive one: a characterization of which combinations of
fairness notions are possible. This work aims to fill this gap by identifying
maximal sets of commonly used fairness measures that can be simultaneously
satisfied. The fairness measures used are demographic parity, equal
opportunity, false positive parity, predictive parity, predictive equality,
overall accuracy equality and treatment equality. We conclude that in total 12
maximal sets of these fairness measures are possible, among which seven
combinations of two measures, and five combinations of three measures. Our work
raises interest questions regarding the practical relevance of each of these 12
maximal fairness notions in various scenarios.

本文研究了 AI 中的公平性问题，探讨了 Fairness measures 可能存在的最大集合以及一些实际应用方面的问题。