With the recent surge and exponential growth of social media usage,
scrutinizing social media content for the presence of any hateful content is of
utmost importance. Researchers have been diligently working since the past
decade on distinguishing between content that promotes hatred and content that
does not. Traditionally, the main focus has been on analyzing textual content.
However, recent research attempts have also commenced into the identification
of audio-based content. Nevertheless, studies have shown that relying solely on
audio or text-based content may be ineffective, as recent upsurge indicates
that individuals often employ sarcasm in their speech and writing. To overcome
these challenges, we present an approach to identify whether a speech promotes
hate or not utilizing both audio and textual representations. Our methodology
is based on the Transformer framework that incorporates both audio and text
sampling, accompanied by our very own layer called "Attentive Fusion". The
results of our study surpassed previous state-of-the-art techniques, achieving
an impressive macro F1 score of 0.927 on the Test Set.

通过结合音频和文本表征，利用 Transformer 方法和自主开发的 “Attentive Fusion” 层，本研究提出了一种识别言论是否宣扬仇恨的方法，取得了 0.927 的宏 F1 得分，超过了此前最先进的技术。

关注融合：一种基于 Transformer 的多模态仇恨言论检测方法

Attentive Fusion: A Transformer-based Approach to Multimodal Hate Speech  Detection

Dense video captioning is a newly emerging task that aims at both localizing
and describing all events in a video. We identify and tackle two challenges on
this task, namely, (1) how to utilize both past and future contexts for
accurate event proposal predictions, and (2) how to construct informative input
to the decoder for generating natural event descriptions. First, previous works
predominantly generate temporal event proposals in the forward direction, which
neglects future video context. We propose a bidirectional proposal method that
effectively exploits both past and future contexts to make proposal
predictions. Second, different events ending at (nearly) the same time are
indistinguishable in the previous works, resulting in the same captions. We
solve this problem by representing each event with an attentive fusion of
hidden states from the proposal module and video contents (e.g., C3D features).
We further propose a novel context gating mechanism to balance the
contributions from the current event and its surrounding contexts dynamically.
We empirically show that our attentively fused event representation is superior
to the proposal hidden states or video contents alone. By coupling proposal and
captioning modules into one unified framework, our model outperforms the
state-of-the-arts on the ActivityNet Captions dataset with a relative gain of
over 100% (Meteor score increases from 4.82 to 9.65).

本研究提出一种双向提案方法、注意力融合和上下文门控机制，将提案与字幕模块结合到一个统一的框架中，以在活动网字幕数据集上实现相对增益超过 100％（Meteor 得分从 4.82 增加到 9.65）的优异表现。