The peaky behavior of CTC models is well known experimentally. However, an
understanding about why peaky behavior occurs is missing, and whether this is a
good property. We provide a formal analysis of the peaky behavior and gradient
descent convergence properties of the CTC loss and related training criteria.
Our analysis provides a deep understanding why peaky behavior occurs and when
it is suboptimal. On a simple example which should be trivial to learn for any
model, we prove that a feed-forward neural network trained with CTC from
uniform initialization converges towards peaky behavior with a 100% error rate.
Our analysis further explains why CTC only works well together with the blank
label. We further demonstrate that peaky behavior does not occur on other
related losses including a label prior model, and that this improves
convergence.

通过分析 CTC 模型的峰形行为，得出了为什么会出现峰形行为以及何时这是不优的深刻认识，并研究了相关的训练标准和梯度下降收敛性。同时发现，CTC 只有与空白标签一起使用才能很好地工作，并证明了其它相关损失不会出现峰形行为从而提高了收敛性。

CTC 为什么会产生尖峰行为？

Why does CTC result in peaky behavior?

A great deal of recent research effort on speech spoofing countermeasures has
been invested into back-end neural networks and training criteria. We
contribute to this effort with a comparative perspective in this study. Our
comparison of countermeasure models on the ASVspoof 2019 logical access task
takes into account recently proposed margin-based training criteria, widely
used front ends, and common strategies to deal with varied-length input trials.
We also measured intra-model differences through multiple training-evaluation
rounds with random initialization. Our statistical analysis demonstrates that
the performance of the same model may be significantly different when just
changing the random initial seed. Thus, we recommend similar analysis or
multiple training-evaluation rounds for further research on the database.
Despite the intra-model differences, we observed a few promising techniques
such as the average pooling to process varied-length inputs and a new
hyper-parameter-free loss function. The two techniques led to the best single
model in our experiment, which achieved an equal error rate of 1.92% and was
significantly different in statistical sense from most of the other
experimental models.

本研究通过对后端神经网络与训练标准的比较，评估 ASVspoof 2019 逻辑访问任务的对策模型，发现随机初始种子会对模型性能造成显著影响，但平均池化和新的无超参数损失函数是比较优秀的技术。

近期神经欺骗对抗合成语音检测方法比较研究

A Comparative Study on Recent Neural Spoofing Countermeasures for  Synthetic Speech Detection

In this paper, a comparative experimental assessment of computer vision-based
methods for sign language recognition is conducted. By implementing the most
recent deep neural network methods in this field, a thorough evaluation on
multiple publicly available datasets is performed. The aim of the present study
is to provide insights on sign language recognition, focusing on mapping
non-segmented video streams to glosses. For this task, two new sequence
training criteria, known from the fields of speech and scene text recognition,
are introduced. Furthermore, a plethora of pretraining schemes is thoroughly
discussed. Finally, a new RGB+D dataset for the Greek sign language is created.
To the best of our knowledge, this is the first sign language dataset where
sentence and gloss level annotations are provided for a video capture.

通过实验评估了基于计算机视觉的手语识别方法，并引入了新的序列训练标准，讨论了许多预训练方案，并创建了带注释的希腊手语 RGB + D 数据集。