Emotion recognition has become a popular topic of interest, especially in the
field of human computer interaction. Previous works involve unimodal analysis
of emotion, while recent efforts focus on multi-modal emotion recognition from
vision and speech. In this paper, we propose a new method of learning about the
hidden representations between just speech and text data using convolutional
attention networks. Compared to the shallow model which employs simple
concatenation of feature vectors, the proposed attention model performs much
better in classifying emotion from speech and text data contained in the
CMU-MOSEI dataset.

本文提出了一种使用卷积注意网络学习语音和文本数据之间隐藏表示的新方法，较之之前的表现更好，能更好地识别出 CMU-MOSEI 数据集中包含的来自语音和文本数据的情感。

用于语音和文本数据的多模态情感识别的卷积注意力网络

Convolutional Attention Networks for Multimodal Emotion Recognition from  Speech and Text Data

Sports channel video portals offer an exciting domain for research on
multimodal, multilingual analysis. We present methods addressing the problem of
automatic video highlight prediction based on joint visual features and textual
analysis of the real-world audience discourse with complex slang, in both
English and traditional Chinese. We present a novel dataset based on League of
Legends championships recorded from North American and Taiwanese Twitch.tv
channels (will be released for further research), and demonstrate strong
results on these using multimodal, character-level CNN-RNN model architectures.

本文提出采用多模态、多语言分析方法预测体育视频集锦的问题，并分析基于联合视觉特征和以实时观众的英文和繁体中文的复杂俚语文本分析的方法。作者基于北美和台湾的 Twitch.tv 频道的英雄联盟冠军系列记录创建了一个新的数据集，并表明了采用基于字符级卷积神经网络（CNN-RNN）模型的强大结果。该数据集将进一步用于研究。

利用观众聊天反应预测视频亮点

Video Highlight Prediction Using Audience Chat Reactions

Studying free-standing conversational groups (FCGs) in unstructured social
settings (e.g., cocktail party ) is gratifying due to the wealth of information
available at the group (mining social networks) and individual (recognizing
native behavioral and personality traits) levels. However, analyzing social
scenes involving FCGs is also highly challenging due to the difficulty in
extracting behavioral cues such as target locations, their speaking activity
and head/body pose due to crowdedness and presence of extreme occlusions. To
this end, we propose SALSA, a novel dataset facilitating multimodal and
Synergetic sociAL Scene Analysis, and make two main contributions to research
on automated social interaction analysis: (1) SALSA records social interactions
among 18 participants in a natural, indoor environment for over 60 minutes,
under the poster presentation and cocktail party contexts presenting
difficulties in the form of low-resolution images, lighting variations,
numerous occlusions, reverberations and interfering sound sources; (2) To
alleviate these problems we facilitate multimodal analysis by recording the
social interplay using four static surveillance cameras and sociometric badges
worn by each participant, comprising the microphone, accelerometer, bluetooth
and infrared sensors. In addition to raw data, we also provide annotations
concerning individuals' personality as well as their position, head, body
orientation and F-formation information over the entire event duration. Through
extensive experiments with state-of-the-art approaches, we show (a) the
limitations of current methods and (b) how the recorded multiple cues
synergetically aid automatic analysis of social interactions. SALSA is
available at this http URL

该研究基于 SALSA 数据集，实现了针对自由对话群体的多模态社交场景分析，该分析可以自动提取每个参与者的行为和个性特点，并论证了多个线索的协同作用如何有助于对社交互动的自动分析。