Diverse human motion prediction aims at predicting multiple possible future
pose sequences from a sequence of observed poses. Previous approaches usually
employ deep generative networks to model the conditional distribution of data,
and then randomly sample outcomes from the distribution. While different
results can be obtained, they are usually the most likely ones which are not
diverse enough. Recent work explicitly learns multiple modes of the conditional
distribution via a deterministic network, which however can only cover a fixed
number of modes within a limited range. In this paper, we propose a novel
sampling strategy for sampling very diverse results from an imbalanced
multimodal distribution learned by a deep generative model. Our method works by
generating an auxiliary space and smartly making randomly sampling from the
auxiliary space equivalent to the diverse sampling from the target
distribution. We propose a simple yet effective network architecture that
implements this novel sampling strategy, which incorporates a Gumbel-Softmax
coefficient matrix sampling method and an aggressive diversity promoting hinge
loss function. Extensive experiments demonstrate that our method significantly
improves both the diversity and accuracy of the samplings compared with
previous state-of-the-art sampling approaches. Code and pre-trained models are
available at this https URL

该文章提出了一种基于 Gumbel-Softmax 采样和 hinge 损失函数，通过在辅助空间内进行随机采样来实现从深度生成模型中多样化采样的新策略，并通过大量实验证明其在人体运动预测方面的效果显著优于现有策略

通过在辅助空间中使用 Gumbel-Softmax 抽样进行多样化的人体动作预测

Diverse Human Motion Prediction via Gumbel-Softmax Sampling from an  Auxiliary Space

It is widely conjectured that the reason that training algorithms for neural
networks are successful because all local minima lead to similar performance,
for example, see (LeCun et al., 2015, Choromanska et al., 2015, Dauphin et al.,
2014). Performance is typically measured in terms of two metrics: training
performance and generalization performance. Here we focus on the training
performance of single-layered neural networks for binary classification, and
provide conditions under which the training error is zero at all local minima
of a smooth hinge loss function. Our conditions are roughly in the following
form: the neurons have to be strictly convex and the surrogate loss function
should be a smooth version of hinge loss. We also provide counterexamples to
show that when the loss function is replaced with quadratic loss or logistic
loss, the result may not hold.

针对单层神经网络的拟合损失函数，研究神经网络算法中局部极小值的性质，提出当神经元是严格凸函数并且代理损失函数是铰链损失函数的平滑版本时，在所有局部极小值处训练误差为零的条件。同时，通过反例表明当损失函数替换为二次损失或逻辑损失时，该结论可能不成立。

神经网络二元分类的损失曲面理解

Understanding the Loss Surface of Neural Networks for Binary  Classification

Fine-grained entity type classification (FETC) is the task of classifying an
entity mention to a broad set of types. Distant supervision paradigm is
extensively used to generate training data for this task. However, generated
training data assigns same set of labels to every mention of an entity without
considering its local context. Existing FETC systems have two major drawbacks:
assuming training data to be noise free and use of hand crafted features. Our
work overcomes both drawbacks. We propose a neural network model that jointly
learns entity mentions and their context representation to eliminate use of
hand crafted features. Our model treats training data as noisy and uses
non-parametric variant of hinge loss function. Experiments show that the
proposed model outperforms previous state-of-the-art methods on two publicly
available datasets, namely FIGER (GOLD) and BBN with an average relative
improvement of 2.69% in micro-F1 score. Knowledge learnt by our model on one
dataset can be transferred to other datasets while using same model or other
FETC systems. These approaches of transferring knowledge further improve the
performance of respective models.

本文提出了一种基于神经网络的模型，联合学习实体提及及其上下文表示，通过使用非参数的变体铰链损失函数解决训练数据含噪的问题，并证明了该模型表现优于现有技术，同时可以实现知识迁移。