Emotion recognition is a core research area at the intersection of artificial
intelligence and human communication analysis. It is a significant technical
challenge since humans display their emotions through complex idiosyncratic
combinations of the language, visual and acoustic modalities. In contrast to
traditional multimodal fusion techniques, we approach emotion recognition from
both direct person-independent and relative person-dependent perspectives. The
direct person-independent perspective follows the conventional emotion
recognition approach which directly infers absolute emotion labels from
observed multimodal features. The relative person-dependent perspective
approaches emotion recognition in a relative manner by comparing partial video
segments to determine if there was an increase or decrease in emotional
intensity. Our proposed model integrates these direct and relative prediction
perspectives by dividing the emotion recognition task into three easier
subtasks. The first subtask involves a multimodal local ranking of relative
emotion intensities between two short segments of a video. The second subtask
uses local rankings to infer global relative emotion ranks with a Bayesian
ranking algorithm. The third subtask incorporates both direct predictions from
observed multimodal behaviors and relative emotion ranks from local-global
rankings for final emotion prediction. Our approach displays excellent
performance on an audio-visual emotion recognition benchmark and improves over
other algorithms for multimodal fusion.

研究了通过综合考虑人员独立和相对依赖角度进行情感识别的方法，该方法使用 Bayesian 排序算法推断全局相对情感排名，最终的情感识别是通过对直接预测和来自本地全局排序的相对情感排名的综合考虑而实现的。