Transformers have become a key architecture in speech processing, but our
understanding of how they build up representations of acoustic and linguistic
structure is limited. In this study, we address this gap by investigating how
measures of 'context-mixing' developed for text models can be adapted and
applied to models of spoken language. We identify a linguistic phenomenon that
is ideal for such a case study: homophony in French (e.g. livre vs livres),
where a speech recognition model has to attend to syntactic cues such as
determiners and pronouns in order to disambiguate spoken words with identical
pronunciations and transcribe them while respecting grammatical agreement. We
perform a series of controlled experiments and probing analyses on
Transformer-based speech models. Our findings reveal that representations in
encoder-only models effectively incorporate these cues to identify the correct
transcription, whereas encoders in encoder-decoder models mainly relegate the
task of capturing contextual dependencies to decoder modules.

通过研究如何将文本模型的 ' 上下文混合 ' 适用于口语语言模型，揭示了 Transformer 模型如何通过关注句法线索来解决同音词消歧问题，并发现编码器与解码器模块在捕捉上下文依赖性任务中的不同表现。

同音词消歧揭示言语变换中上下文混合的模式

Homophone Disambiguation Reveals Patterns of Context Mixing in Speech  Transformers

Homophony's widespread presence in natural languages is a controversial
topic. Recent theories of language optimality have tried to justify its
prevalence, despite its negative effects on cognitive processing time; e.g.,
Piantadosi et al. (2012) argued homophony enables the reuse of efficient
wordforms and is thus beneficial for languages. This hypothesis has recently
been challenged by Trott and Bergen (2020), who posit that good wordforms are
more often homophonous simply because they are more phonotactically probable.
In this paper, we join in on the debate. We first propose a new
information-theoretic quantification of a language's homophony: the sample
R\'enyi entropy. Then, we use this quantification to revisit Trott and Bergen's
claims. While their point is theoretically sound, a specific methodological
issue in their experiments raises doubts about their results. After addressing
this issue, we find no clear pressure either towards or against homophony -- a
much more nuanced result than either Piantadosi et al.'s or Trott and Bergen's
findings.

本文对自然语言中同音现象的普遍性和其对认知加工时间的负面影响进行了讨论，并通过新的信息熵度量方法，重新审视了语言优化理论及其对于同音现象的解释，最终发现无法得出一致的结论。