While text-based event extraction has been an active research area and has
seen successful application in many domains, extracting semantic events from
speech directly is an under-explored problem. In this paper, we introduce the
Speech Event Extraction (SpeechEE) task and construct three synthetic training
sets and one human-spoken test set. Compared to event extraction from text,
SpeechEE poses greater challenges mainly due to complex speech signals that are
continuous and have no word boundaries. Additionally, unlike perceptible sound
events, semantic events are more subtle and require a deeper understanding. To
tackle these challenges, we introduce a sequence-to-structure generation
paradigm that can produce events from speech signals in an end-to-end manner,
together with a conditioned generation method that utilizes speech recognition
transcripts as the contextual clue. We further propose to represent events with
a flat format to make outputs more natural language-like. Our experimental
results show that our method brings significant improvements on all datasets,
achieving a maximum F1 gain of 10.7%. The code and datasets are released on
this https URL

我们介绍了一种从语音信号中提取事件的新方法 ——Speech Event Extraction (SpeechEE)。我们提出了一种序列到结构生成范例，可以在端到端的方式上从语音信号中生成事件，并利用语音识别转录作为上下文线索的生成方法。实验结果显示，我们的方法在所有数据集上都取得了显著的改进，最大 F1 增益达到了 10.7%。