Automatic subtitling is the task of automatically translating the speech of
an audiovisual product into short pieces of timed text, in other words,
subtitles and their corresponding timestamps. The generated subtitles need to
conform to multiple space and time requirements (length, reading speed) while
being synchronised with the speech and segmented in a way that facilitates
comprehension. Given its considerable complexity, automatic subtitling has so
far been addressed through a pipeline of elements that deal separately with
transcribing, translating, segmenting into subtitles and predicting timestamps.
In this paper, we propose the first direct automatic subtitling model that
generates target language subtitles and their timestamps from the source speech
in a single solution. Comparisons with state-of-the-art cascaded models trained
with both in- and out-domain data show that our system provides high-quality
subtitles while also being competitive in terms of conformity, with all the
advantages of maintaining a single model.

本论文提出了一种直接自动生成目标语言字幕及相应时间戳的自动字幕模型，与现有的级联模型相比，在提供高质量字幕的同时也具有更高的一致性和维护单一模型的优点。

自动字幕直接语音翻译

Direct Speech Translation for Automatic Subtitling

Movies provide us with a mass of visual content as well as attracting
stories. Existing methods have illustrated that understanding movie stories
through only visual content is still a hard problem. In this paper, for
answering questions about movies, we put forward a Layered Memory Network (LMN)
that represents frame-level and clip-level movie content by the Static Word
Memory module and the Dynamic Subtitle Memory module, respectively.
Particularly, we firstly extract words and sentences from the training movie
subtitles. Then the hierarchically formed movie representations, which are
learned from LMN, not only encode the correspondence between words and visual
content inside frames, but also encode the temporal alignment between sentences
and frames inside movie clips. We also extend our LMN model into three variant
frameworks to illustrate the good extendable capabilities. We conduct extensive
experiments on the MovieQA dataset. With only visual content as inputs, LMN
with frame-level representation obtains a large performance improvement. When
incorporating subtitles into LMN to form the clip-level representation, we
achieve the state-of-the-art performance on the online evaluation task of
'Video+Subtitles'. The good performance successfully demonstrates that the
proposed framework of LMN is effective and the hierarchically formed movie
representations have good potential for the applications of movie question
answering.

本论文提出采用分层记忆网络来对电影进行理解，利用静态词记忆模块以及动态字幕记忆模块来表示帧级和片段级的电影内容，成功提高了电影问题回答任务的效果，证明了分层记忆网络的有效性和电影表示形式的潜力。