Originally introduced as a neural network for ensemble learning, mixture of
experts (MoE) has recently become a fundamental building block of highly
successful modern deep neural networks for heterogeneous data analysis in
several applications, including those in machine learning, statistics,
bioinformatics, economics, and medicine. Despite its popularity in practice, a
satisfactory level of understanding of the convergence behavior of
Gaussian-gated MoE parameter estimation is far from complete. The underlying
reason for this challenge is the inclusion of covariates in the Gaussian gating
and expert networks, which leads to their intrinsically complex interactions
via partial differential equations with respect to their parameters. We address
these issues by designing novel Voronoi loss functions to accurately capture
heterogeneity in the maximum likelihood estimator (MLE) for resolving parameter
estimation in these models. Our results reveal distinct behaviors of the MLE
under two settings: the first setting is when all the location parameters in
the Gaussian gating are non-zeros while the second setting is when there exists
at least one zero-valued location parameter. Notably, these behaviors can be
characterized by the solvability of two different systems of polynomial
equations. Finally, we conduct a simulation study to verify our theoretical
results.

本研究利用 Voronoi 损失函数来解决高斯门限混合专家模型参数估计中出现的内在复杂交互问题。在两种不同的场景下，对于 MLE 行为进行了分析，并最终进行了模拟研究以验证理论结果。