In this study, we present synchronous bilingual Connectionist Temporal
Classification (CTC), an innovative framework that leverages dual CTC to bridge
the gaps of both modality and language in the speech translation (ST) task.
Utilizing transcript and translation as concurrent objectives for CTC, our
model bridges the gap between audio and text as well as between source and
target languages. Building upon the recent advances in CTC application, we
develop an enhanced variant, BiL-CTC+, that establishes new state-of-the-art
performances on the MuST-C ST benchmarks under resource-constrained scenarios.
Intriguingly, our method also yields significant improvements in speech
recognition performance, revealing the effect of cross-lingual learning on
transcription and demonstrating its broad applicability. The source code is
available at this https URL

本研究提出了一种创新的同步双语联结主义时间分类 (CTC) 框架，利用双重 CTC 来填补语音翻译任务中模态和语言之间的差距。通过将转录和翻译作为 CTC 的并行目标，我们的模型填补了音频和文本以及源语言和目标语言之间的差距。在 CTC 应用的最新进展基础上，我们还开发了一个增强的变体 BiL-CTC+，在资源受限场景下创造出了新的最先进性能。有趣的是，我们的方法还显著提高了语音识别性能，揭示了跨语言学习对转录的影响，并展示了其广泛的适用性。源代码可以在此 https URL 获取。