The paper presents a method for spoken term detection based on the
Transformer architecture. We propose the encoder-encoder architecture employing
two BERT-like encoders with additional modifications, including convolutional
and upsampling layers, attention masking, and shared parameters. The encoders
project a recognized hypothesis and a searched term into a shared embedding
space, where the score of the putative hit is computed using the calibrated dot
product. In the experiments, we used the Wav2Vec 2.0 speech recognizer, and the
proposed system outperformed a baseline method based on deep LSTMs on the
English and Czech STD datasets based on USC Shoah Foundation Visual History
Archive (MALACH).

本文提出了一种基于 Transformer 结构的语音词项检测方法，采用两个 BERT-like 编码器并进行了适当的修改，包括卷积和上采样层、注意力掩码、共享参数等，将假设和搜索术语投影到共享嵌入空间中，并使用校准点积计算假设的命中得分。在实验中，使用了 Wav2Vec 2.0 语音识别器，并在基于 USC Shoah 基金会视觉历史档案（MALACH）的英语和捷克 STD 数据集上，提出的系统优于基于深度 LSTM 的基线方法。

基于 Transformer 编码器 - 编码器架构的口语检索

Transformer-based encoder-encoder architecture for Spoken Term Detection

The paper describes a novel approach to Spoken Term Detection (STD) in large
spoken archives using deep LSTM networks. The work is based on the previous
approach of using Siamese neural networks for STD and naturally extends it to
directly localize a spoken term and estimate its relevance score. The phoneme
confusion network generated by a phoneme recognizer is processed by the deep
LSTM network which projects each segment of the confusion network into an
embedding space. The searched term is projected into the same embedding space
using another deep LSTM network. The relevance score is then computed using a
simple dot-product in the embedding space and calibrated using a sigmoid
function to predict the probability of occurrence. The location of the searched
term is then estimated from the sequence of output probabilities. The deep LSTM
networks are trained in a self-supervised manner from paired recognition
hypotheses on word and phoneme levels. The method is experimentally evaluated
on MALACH data in English and Czech languages.

该论文提出了一种利用深度 LSTM 网络在大型口语档案中进行口语检测的新方法，通过对语音识别器生成的音素混淆网络进行处理，并使用另一个深度 LSTM 网络来将搜索术语投影到同一嵌入空间中，计算相关度，并使用 sigmod 函数进行校准来预测发生的概率，最后从输出概率序列中估计搜索术语的位置。

利用发音嵌入的点积进行口语术语识别和相关分数估计

Spoken Term Detection and Relevance Score Estimation using Dot-Product of Pronunciation Embeddings

We propose a novel transcription workflow which combines spoken term
detection and human-in-the-loop, together with a pilot experiment. This work is
grounded in an almost zero-resource scenario where only a few terms have so far
been identified, involving two endangered languages. We show that in the early
stages of transcription, when the available data is insufficient to train a
robust ASR system, it is possible to take advantage of the transcription of a
small number of isolated words in order to bootstrap the transcription of a
speech collection.

我们提出了一种新的转录工作流，将口语符号检测和人工参与相结合，以及一项试验。在只有少数项已被识别的几乎零资源情景中，涉及两种濒危语言。我们展示了在转录工作初期，当可用数据数量不足以训练鲁棒 ASR 系统时，可以利用少量孤立的单词的转录来引导语音收集的转录。