An ASR system usually does not predict any punctuation or capitalization.
Lack of punctuation causes problems in result presentation and confuses both
the human reader andoff-the-shelf natural language processing algorithms. To
overcome these limitations, we train two variants of Deep Neural Network (DNN)
sequence labelling models - a Bidirectional Long Short-Term Memory (BLSTM) and
a Convolutional Neural Network (CNN), to predict the punctuation. The models
are trained on the Fisher corpus which includes punctuation annotation. In our
experiments, we combine time-aligned and punctuated Fisher corpus transcripts
using a sequence alignment algorithm. The neural networks are trained on Common
Web Crawl GloVe embedding of the words in Fisher transcripts aligned with
conversation side indicators and word time infomation. The CNNs yield a better
precision and BLSTMs tend to have better recall. While BLSTMs make fewer
mistakes overall, the punctuation predicted by the CNN is more accurate -
especially in the case of question marks. Our results constitute significant
evidence that the distribution of words in time, as well as pre-trained
embeddings, can be useful in the punctuation prediction task.

本文通过训练基于深度神经网络技术的双向长短期记忆网络（BLSTM）和卷积神经网络（CNN）的序列标注模型，来解决语音识别系统中无法正常预测标点符号的问题，并且通过在 Fisher 语料库上的实验证明，基于卷积神经网络预测标点符号的方法更加精确。同时，本文结果表明，将 Fisher 语料库转化为时间对齐的文本以及采用预训练嵌入模型可以提高标点预测的准确性。

面向会话语音的标点符号预测模型

Punctuation Prediction Model for Conversational Speech

Most existing sequence labelling models rely on a fixed decomposition of a
target sequence into a sequence of basic units. These methods suffer from two
major drawbacks: 1) the set of basic units is fixed, such as the set of words,
characters or phonemes in speech recognition, and 2) the decomposition of
target sequences is fixed. These drawbacks usually result in sub-optimal
performance of modeling sequences. In this pa- per, we extend the popular CTC
loss criterion to alleviate these limitations, and propose a new loss function
called Gram-CTC. While preserving the advantages of CTC, Gram-CTC automatically
learns the best set of basic units (grams), as well as the most suitable
decomposition of tar- get sequences. Unlike CTC, Gram-CTC allows the model to
output variable number of characters at each time step, which enables the model
to capture longer term dependency and improves the computational efficiency. We
demonstrate that the proposed Gram-CTC improves CTC in terms of both
performance and efficiency on the large vocabulary speech recognition task at
multiple scales of data, and that with Gram-CTC we can outperform the
state-of-the-art on a standard speech benchmark.

本文介绍一种基于 CTC loss criterion 的新的损失函数 Gram-CTC，它在语音识别等任务上取得了比 CTC 更好的性能和效率，并能捕获更长期的依赖关系。