Machine learning has begun to play a central role in many applications. A
multitude of these applications typically also involve datasets that are
distributed across multiple computing devices/machines due to either design
constraints (e.g., multiagent systems) or computational/privacy reasons (e.g.,
learning on smartphone data). Such applications often require the learning
tasks to be carried out in a decentralized fashion, in which there is no
central server that is directly connected to all nodes. In real-world
decentralized settings, nodes are prone to undetected failures due to
malfunctioning equipment, cyberattacks, etc., which are likely to crash
non-robust learning algorithms. The focus of this paper is on robustification
of decentralized learning in the presence of nodes that have undergone
Byzantine failures. The Byzantine failure model allows faulty nodes to
arbitrarily deviate from their intended behaviors, thereby ensuring designs of
the most robust of algorithms. But the study of Byzantine resilience within
decentralized learning, in contrast to distributed learning, is still in its
infancy. In particular, existing Byzantine-resilient decentralized learning
methods either do not scale well to large-scale machine learning models, or
they lack statistical convergence guarantees that help characterize their
generalization errors. In this paper, a scalable, Byzantine-resilient
decentralized machine learning framework termed Byzantine-resilient
decentralized gradient descent (BRIDGE) is introduced. Algorithmic and
statistical convergence guarantees for one variant of BRIDGE are also provided
in the paper for both strongly convex problems and a class of nonconvex
problems. In addition, large-scale decentralized learning experiments are used
to establish that the BRIDGE framework is scalable and it delivers competitive
results for Byzantine-resilient convex and nonconvex learning.

本文介绍了一种可扩展的拜占庭容错分布式机器学习框架（BRIDGE），其提供了一些算法收敛和统计收敛保证，适用于强凸问题和一类非凸问题，并在大规模分散学习实验中证明其可扩展性和实用性。

BRIDGE: 拜占庭容错的分布式梯度下降算法

BRIDGE: Byzantine-resilient Decentralized Gradient Descent

Neural networks are usually not the tool of choice for nonparametric
high-dimensional problems where the number of input features is much larger
than the number of observations. Though neural networks can approximate complex
multivariate functions, they generally require a large number of training
observations to obtain reasonable fits, unless one can learn the appropriate
network structure. In this manuscript, we show that neural networks can be
applied successfully to high-dimensional settings if the true function falls in
a low dimensional subspace, and proper regularization is used. We propose
fitting a neural network with a sparse group lasso penalty on the first-layer
input weights. This results in a neural net that only uses a small subset of
the original features. In addition, we characterize the statistical convergence
of the penalized empirical risk minimizer to the optimal neural network: we
show that the excess risk of this penalized estimator only grows with the
logarithm of the number of input features; and we show that the weights of
irrelevant features converge to zero. Via simulation studies and data analyses,
we show that these sparse-input neural networks outperform existing
nonparametric high-dimensional estimation methods when the data has complex
higher-order interactions.

该论文提出了使用稀疏组套索惩罚来适应神经网络以解决非参数高维问题，其中真实功能位于低维子空间中，并对统计收敛性进行了表征。

高维非参数回归和分类的稀疏输入神经网络

Sparse-Input Neural Networks for High-dimensional Nonparametric  Regression and Classification

In this article, we study the question of the statistical convergence of the
1-dimensional Mapper to its continuous analogue, the Reeb graph. We show that
the Mapper is an optimal estimator of the Reeb graph, which gives, as a
byproduct, a method to automatically tune its parameters and compute confidence
regions on its topological features, such as its loops and flares. This allows
to circumvent the issue of testing a large grid of parameters and keeping the
most stable ones in the brute-force setting, which is widely used in
visualization, clustering and feature selection with the Mapper.

本文研究了一维 Mapper 收敛于其连续模拟 - Reeb 图的统计性问题。我们证明了 Mapper 是 Reeb 图的最优估计器，其可以自动调整参数并计算其拓扑特征的置信区间。这可避免在 Mapper 的可视化、聚类和特征选择中使用大网格的参数测试和保留最稳定实践的问题。