Existing top-performance 3D object detectors typically rely on the
multi-modal fusion strategy. This design is however fundamentally restricted
due to overlooking the modality-specific useful information and finally
hampering the model performance. To address this limitation, in this work we
introduce a novel modality interaction strategy where individual per-modality
representations are learned and maintained throughout for enabling their unique
characteristics to be exploited during object detection. To realize this
proposed strategy, we design a DeepInteraction architecture characterized by a
multi-modal representational interaction encoder and a multi-modal predictive
interaction decoder. Experiments on the large-scale nuScenes dataset show that
our proposed method surpasses all prior arts often by a large margin.
Crucially, our method is ranked at the first position at the highly competitive
nuScenes object detection leaderboard.

本文提出一种新的模态交互策略，采用 DeepInteraction 结构对多模态特征进行交互和预测，此方法在大规模 nuScenes 数据集上取得比之前所有技术更好的结果，并跻身 nuScenes 目标检测榜首。

DeepInteraction: 通过模态交互进行 3D 物体检测

DeepInteraction: 3D Object Detection via Modality Interaction

Continuous dimensional emotion prediction is a challenging task where the
fusion of various modalities usually achieves state-of-the-art performance such
as early fusion or late fusion. In this paper, we propose a novel multi-modal
fusion strategy named conditional attention fusion, which can dynamically pay
attention to different modalities at each time step. Long-short term memory
recurrent neural networks (LSTM-RNN) is applied as the basic uni-modality model
to capture long time dependencies. The weights assigned to different modalities
are automatically decided by the current input features and recent history
information rather than being fixed at any kinds of situation. Our experimental
results on a benchmark dataset AVEC2015 show the effectiveness of our method
which outperforms several common fusion strategies for valence prediction.

该研究提出了一种名为条件注意融合的新型多模态融合策略，将长短期记忆循环神经网络应用为基础单模态模型，自动决定分配给不同情感模态的权重，获得了在情感价值预测方面优于多种常见融合策略的实验结果。