In this paper, we study how word-like units are represented and activated in
a recurrent neural model of visually grounded speech. The model used in our
experiments is trained to project an image and its spoken description in a
common representation space. We show that a recurrent model trained on spoken
sentences implicitly segments its input into word-like units and reliably maps
them to their correct visual referents. We introduce a methodology originating
from linguistics to analyse the representation learned by neural networks --
the gating paradigm -- and show that the correct representation of a word is
only activated if the network has access to first phoneme of the target word,
suggesting that the network does not rely on a global acoustic pattern.
Furthermore, we find out that not all speech frames (MFCC vectors in our case)
play an equal role in the final encoded representation of a given word, but
that some frames have a crucial effect on it. Finally, we suggest that word
representation could be activated through a process of lexical competition.

本文研究了如何在视觉语音循环神经网络中表示和识别单词，使用门控方法分析神经网络所学单词表征的正确性和词汇竞争过程中的单词激活方式，其结果表明当网络能接触到目标单词第一个音素时才能正确识别单词表征。