Subtitling plays a crucial role in enhancing the accessibility of audiovisual
content and encompasses three primary subtasks: translating spoken dialogue,
segmenting translations into concise textual units, and estimating timestamps
that govern their on-screen duration. Past attempts to automate this process
rely, to varying degrees, on automatic transcripts, employed diversely for the
three subtasks. In response to the acknowledged limitations associated with
this reliance on transcripts, recent research has shifted towards
transcription-free solutions for translation and segmentation, leaving the
direct generation of timestamps as uncharted territory. To fill this gap, we
introduce the first direct model capable of producing automatic subtitles,
entirely eliminating any dependence on intermediate transcripts also for
timestamp prediction. Experimental results, backed by manual evaluation,
showcase our solution's new state-of-the-art performance across multiple
language pairs and diverse conditions.

直接生成的自动字幕模型填补了依赖于中间文件的不足，通过消除对中间转录的依赖，实现了对多语种、多样化环境下的自动字幕新的最佳性能。

自动字幕中消除对剧本依赖性的 SBAAM 方法

SBAAM! Eliminating Transcript Dependency in Automatic Subtitling

Speech translation for subtitling (SubST) is the task of automatically
translating speech data into well-formed subtitles by inserting subtitle breaks
compliant to specific displaying guidelines. Similar to speech translation
(ST), model training requires parallel data comprising audio inputs paired with
their textual translations. In SubST, however, the text has to be also
annotated with subtitle breaks. So far, this requirement has represented a
bottleneck for system development, as confirmed by the dearth of publicly
available SubST corpora. To fill this gap, we propose a method to convert
existing ST corpora into SubST resources without human intervention. We build a
segmenter model that automatically segments texts into proper subtitles by
exploiting audio and text in a multimodal fashion, achieving high segmentation
quality in zero-shot conditions. Comparative experiments with SubST systems
respectively trained on manual and automatic segmentations result in similar
performance, showing the effectiveness of our approach.

本研究提出了一种利用多模态方法将现有的语音翻译资料转换为符合 SubST 准则的字幕翻译资料的方法。通过训练一个基于音频和文本结合的分段模型，实现高质量的自动分段，并通过对比实验验证了该方法的有效性。