Acoustic word embeddings (AWEs) are fixed-dimensional vector representations
of speech segments that encode phonetic content so that different realisations
of the same word have similar embeddings. In this paper we explore semantic AWE
modelling. These AWEs should not only capture phonetics but also the meaning of
a word (similar to textual word embeddings). We consider the scenario where we
only have untranscribed speech in a target language. We introduce a number of
strategies leveraging a pre-trained multilingual AWE model -- a phonetic AWE
model trained on labelled data from multiple languages excluding the target.
Our best semantic AWE approach involves clustering word segments using the
multilingual AWE model, deriving soft pseudo-word labels from the cluster
centroids, and then training a Skipgram-like model on the soft vectors. In an
intrinsic word similarity task measuring semantics, this multilingual transfer
approach outperforms all previous semantic AWE methods. We also show -- for the
first time -- that AWEs can be used for downstream semantic query-by-example
search.

本文研究了声学词嵌入，探索了语义学声学词嵌入，并提出了一种基于多语种转移的语义学声学词嵌入方法，能够显著提高词汇相似性任务的表现，并且能够用于语义检索。

利用多语言迁移学习进行无监督语义声学词嵌入

Leveraging multilingual transfer for unsupervised semantic acoustic word  embeddings

Query-by-example search often uses dynamic time warping (DTW) for comparing
queries and proposed matching segments. Recent work has shown that comparing
speech segments by representing them as fixed-dimensional vectors --- acoustic
word embeddings --- and measuring their vector distance (e.g., cosine distance)
can discriminate between words more accurately than DTW-based approaches. We
consider an approach to query-by-example search that embeds both the query and
database segments according to a neural model, followed by nearest-neighbor
search to find the matching segments. Earlier work on embedding-based
query-by-example, using template-based acoustic word embeddings, achieved
competitive performance. We find that our embeddings, based on recurrent neural
networks trained to optimize word discrimination, achieve substantial
improvements in performance and run-time efficiency over the previous
approaches.

使用神经模型将查询语音和数据库语音嵌入后进行最近邻搜索，相比模板式语音嵌入取得了更高的性能和运行效率。

使用鉴别性神经声学单词嵌入的示例查询搜索

Query-by-Example Search with Discriminative Neural Acoustic Word  Embeddings

Acoustic word embeddings --- fixed-dimensional vector representations of
variable-length spoken word segments --- have begun to be considered for tasks
such as speech recognition and query-by-example search. Such embeddings can be
learned discriminatively so that they are similar for speech segments
corresponding to the same word, while being dissimilar for segments
corresponding to different words. Recent work has found that acoustic word
embeddings can outperform dynamic time warping on query-by-example search and
related word discrimination tasks. However, the space of embedding models and
training approaches is still relatively unexplored. In this paper we present
new discriminative embedding models based on recurrent neural networks (RNNs).
We consider training losses that have been successful in prior work, in
particular a cross entropy loss for word classification and a contrastive loss
that explicitly aims to separate same-word and different-word pairs in a
"Siamese network" training setting. We find that both classifier-based and
Siamese RNN embeddings improve over previously reported results on a word
discrimination task, with Siamese RNNs outperforming classification models. In
addition, we present analyses of the learned embeddings and the effects of
variables such as dimensionality and network structure.

本研究基于递归神经网络 (RNN) 提出了新的区分性嵌入模型，探讨了不同的训练损失，通过单词区分任务的评估发现，相比以往的模型，分类器和 Siamese RNNs 都有所改进，其中 Siamese RNNs 的表现更为优异，同时也对嵌入模型和网络结构等变量进行了分析。