Punctuation restoration plays an essential role in the post-processing
procedure of automatic speech recognition, but model efficiency is a key
requirement for this task. To that end, we present EfficientPunct, an ensemble
method with a multimodal time-delay neural network that outperforms the current
best model by 1.0 F1 points, using less than a tenth of its parameters to
process embeddings. We streamline a speech recognizer to efficiently output
hidden layer latent vectors as audio embeddings for punctuation restoration, as
well as BERT to extract meaningful text embeddings. By using forced alignment
and temporal convolutions, we eliminate the need for multi-head attention-based
fusion, greatly increasing computational efficiency but also raising
performance. EfficientPunct sets a new state of the art, in terms of both
performance and efficiency, with an ensemble that weights BERT's purely
language-based predictions slightly more than the multimodal network's
predictions.

EfficientPunct 使用多模时间延迟神经网络实现标点符号的恢复，它在性能和效率方面都创造了新的最佳结果，其中使用 BERT 纯文本嵌入的预测的权重略高于多模网络的预测。