Speech emotion recognition (SER) has received a great deal of attention in
recent years in the context of spontaneous conversations. While there have been
notable results on datasets like the well known corpus of naturalistic dyadic
conversations, IEMOCAP, for both the case of categorical and dimensional
emotions, there are few papers which try to predict both paradigms at the same
time. Therefore, in this work, we aim to highlight the performance contribution
of multi-task learning by proposing a multi-task, multi-modal system that
predicts categorical and dimensional emotions. The results emphasise the
importance of cross-regularisation between the two types of emotions. Our
approach consists of a multi-task, multi-modal architecture that uses parallel
feature refinement through self-attention for the feature of each modality. In
order to fuse the features, our model introduces a set of learnable bridge
tokens that merge the acoustic and linguistic features with the help of
cross-attention. Our experiments for categorical emotions on 10-fold validation
yield results comparable to the current state-of-the-art. In our configuration,
our multi-task approach provides better results compared to learning each
paradigm separately. On top of that, our best performing model achieves a high
result for valence compared to the previous multi-task experiments.

多任务学习在语音情绪识别中的应用与多模态系统和情绪范畴及情绪维度的预测相关，通过跨任务正则化和特征融合提高模型表现。

多任务多模态预测分类和维度情感

A Multi-Task, Multi-Modal Approach for Predicting Categorical and  Dimensional Emotions

Conversational text-to-speech (TTS) aims to synthesize speech with proper
prosody of reply based on the historical conversation. However, it is still a
challenge to comprehensively model the conversation, and a majority of
conversational TTS systems only focus on extracting global information and omit
local prosody features, which contain important fine-grained information like
keywords and emphasis. Moreover, it is insufficient to only consider the
textual features, and acoustic features also contain various prosody
information. Hence, we propose M2-CTTS, an end-to-end multi-scale multi-modal
conversational text-to-speech system, aiming to comprehensively utilize
historical conversation and enhance prosodic expression. More specifically, we
design a textual context module and an acoustic context module with both
coarse-grained and fine-grained modeling. Experimental results demonstrate that
our model mixed with fine-grained context information and additionally
considering acoustic features achieves better prosody performance and
naturalness in CMOS tests.

提出了一种多尺度，多模态会话文本到语音系统（M2-CTTS），用于综合利用历史会话并增强韵律表达，通过考虑文本和声学因素的粗粒度和细粒度建模，并混合细粒度上下文信息及声学特征，实现了更好的韵律表现和自然度。

M2-CTTS: 端到端的多尺度、多模态会话文本到语音合成

M2-CTTS: End-to-End Multi-scale Multi-modal Conversational  Text-to-Speech Synthesis

Comprehensive robustness analysis of PECNet, a pedestrian trajectory
prediction system for autonomous vehicles. A novel metric is introduced for
dataset analysis and classification. Synthetic data augmentation techniques
ranging from Newtonian mechanics to Deep Reinforcement Learning based
simulations are used to improve and test the system. An improvement of 9.5%
over state-of-the-art results is seen on the FDE while compromising ADE. We
introduce novel architectural changes using SIRENs for higher precision results
to validate our robustness hypotheses. Additionally, we diagrammatically
propose a novel multi-modal system for the same task.

本文介绍了用于自动驾驶车辆的行人轨迹预测系统 PECNet 的全面健壮性分析，使用从牛顿力学到基于深度强化学习的模拟的合成数据增强技术来提高和测试系统，提出了一种新的度量标准来进行数据集分析和分类，并在采用 SIRENs 的新体系结构的情况下提高了 9.5％的性能。同时我们提出了一种多模式的解决方案。