In modern machine learning problems we deal with datasets that are either
distributed by nature or potentially large for which distributing the
computations is usually a standard way to proceed, since centralized algorithms
are in general ineffective. We propose a distributed learning approach for
mixtures of experts (MoE) models with an aggregation strategy to construct a
reduction estimator from local estimators fitted parallelly to distributed
subsets of the data. The aggregation is based on an optimal minimization of an
expected transportation divergence between the large MoE composed of local
estimators and the unknown desired MoE model. We show that the provided
reduction estimator is consistent as soon as the local estimators to be
aggregated are consistent, and its construction is performed by a proposed
majorization-minimization (MM) algorithm that is computationally effective. We
study the statistical and numerical properties for the proposed reduction
estimator on experiments that demonstrate its performance compared to namely
the global estimator constructed in a centralized way from the full dataset.
For some situations, the computation time is more than ten times faster, for a
comparable performance. Our source codes are publicly available on Github.

提出了一种分布式学习方法，用于构建一个由本地估计器并行拟合数据子集所组成的大的混合专家模型，通过最小化期望的运输散度来聚合这些本地估计器，并通过提出的主导 - 最小化算法来构造一种计算规模高效的降维估计器。对实验中的统计和数值属性进行了研究，证明了所提供的降维估计器的性能优于从完整数据集中以集中方式构造的全局估计器，有些情况下计算时间比全局估计器快十倍以上，我们的源代码公开在 Github 上。