Multi-modal regression is important in forecasting nonstationary processes or
with a complex mixture of distributions. It can be tackled with multiple
hypotheses frameworks but with the difficulty of combining them efficiently in
a learning model. A Structured Radial Basis Function Network is presented as an
ensemble of multiple hypotheses predictors for regression problems. The
predictors are regression models of any type that can form centroidal Voronoi
tessellations which are a function of their losses during training. It is
proved that this structured model can efficiently interpolate this tessellation
and approximate the multiple hypotheses target distribution and is equivalent
to interpolating the meta-loss of the predictors, the loss being a zero set of
the interpolation error. This model has a fixed-point iteration algorithm
between the predictors and the centers of the basis functions. Diversity in
learning can be controlled parametrically by truncating the tessellation
formation with the losses of individual predictors. A closed-form solution with
least-squares is presented, which to the authors knowledge, is the fastest
solution in the literature for multiple hypotheses and structured predictions.
Superior generalization performance and computational efficiency is achieved
using only two-layer neural networks as predictors controlling diversity as a
key component of success. A gradient-descent approach is introduced which is
loss-agnostic regarding the predictors. The expected value for the loss of the
structured model with Gaussian basis functions is computed, finding that
correlation between predictors is not an appropriate tool for diversification.
The experiments show outperformance with respect to the top competitors in the
literature.

多模态回归问题可以通过结构化径向基函数网络来解决，该网络将多个假设预测器集成在一起，通过训练期间损失函数的 Loss 值形成质心 Voronoi tessellations，从而高效地插值这些 tessellations 并逼近多个假设的目标分布，同时使用只有两层神经网络的预测器控制多样性，达到了卓越的泛化性能和计算效率。

结构化径向基函数网络：多个假设预测的多样性建模

Structured Radial Basis Function Network: Modelling Diversity for  Multiple Hypotheses Prediction

The recent introduction of Graph Neural Networks (GNNs) and their growing
popularity in the past few years has enabled the application of deep learning
algorithms to non-Euclidean, graph-structured data. GNNs have achieved
state-of-the-art results across an impressive array of graph-based machine
learning problems. Nevertheless, despite their rapid pace of development, much
of the work on GNNs has focused on graph classification and embedding
techniques, largely ignoring regression tasks over graph data. In this paper,
we develop a Graph Mixture Density Network (GraphMDN), which combines graph
neural networks with mixture density network (MDN) outputs. By combining these
techniques, GraphMDNs have the advantage of naturally being able to incorporate
graph structured information into a neural architecture, as well as the ability
to model multi-modal regression targets. As such, GraphMDNs are designed to
excel on regression tasks wherein the data are graph structured, and target
statistics are better represented by mixtures of densities rather than singular
values (so-called ``inverse problems"). To demonstrate this, we extend an
existing GNN architecture known as Semantic GCN (SemGCN) to a GraphMDN
structure, and show results from the Human3.6M pose estimation task. The
extended model consistently outperforms both GCN and MDN architectures on their
own, with a comparable number of parameters.

本文研究了新型的图神经网络 (Graph Neural Networks) 在处理非欧几里德图结构数据时的应用。作者提出了一种将 Graph Neural Network 和混合密度网络 (Mixture Density Network, MDN) 相结合的新型网络 (Graph Mixture Density Network), 该网络可将图结构信息嵌入到深度神经网络中，并在数据具有图结构、有多模态回归目标或是需要模拟多峰分布的情况下表现优异。作者在基于人类姿势估计的任务中，比较了该网络与其他现有的图神经网络模型，结果表明该模型有效地提升了预测性能。