While LiDAR sensors have been succesfully applied to 3D object detection, the
affordability of radar and camera sensors has led to a growing interest in
fusiong radars and cameras for 3D object detection. However, previous
radar-camera fusion models have not been able to fully utilize radar
information in that initial 3D proposals were generated based on the camera
features only and the instance-level fusion is subsequently conducted. In this
paper, we propose radar-camera multi-level fusion (RCM-Fusion), which fuses
radar and camera modalities at both the feature-level and instance-level to
fully utilize radar information. At the feature-level, we propose a Radar
Guided BEV Encoder which utilizes radar Bird's-Eye-View (BEV) features to
transform image features into precise BEV representations and then adaptively
combines the radar and camera BEV features. At the instance-level, we propose a
Radar Grid Point Refinement module that reduces localization error by
considering the characteristics of the radar point clouds. The experiments
conducted on the public nuScenes dataset demonstrate that our proposed
RCM-Fusion offers 11.8% performance gain in nuScenes detection score (NDS) over
the camera-only baseline model and achieves state-of-the-art performaces among
radar-camera fusion methods in the nuScenes 3D object detection benchmark. Code
will be made publicly available.

本文提出了雷达 - 相机多层融合（RCM-Fusion）方法，通过特征级和实例级融合方法充分利用雷达信息，实验结果在公开的 NuScenes 数据集上表明我们的方法在 NuScenes 三维物体检测基准上取得了最先进的性能，检测分数（NDS）较仅相机方法提高了 11.8%。

RCM-Fusion：雷达 - 相机多级融合的三维物体检测

RCM-Fusion: Radar-Camera Multi-Level Fusion for 3D Object Detection

Multimodal Knowledge Graphs (MKGs), which organize visual-text factual
knowledge, have recently been successfully applied to tasks such as information
retrieval, question answering, and recommendation system. Since most MKGs are
far from complete, extensive knowledge graph completion studies have been
proposed focusing on the multimodal entity, relation extraction and link
prediction. However, different tasks and modalities require changes to the
model architecture, and not all images/objects are relevant to text input,
which hinders the applicability to diverse real-world scenarios. In this paper,
we propose a hybrid transformer with multi-level fusion to address those
issues. Specifically, we leverage a hybrid transformer architecture with
unified input-output for diverse multimodal knowledge graph completion tasks.
Moreover, we propose multi-level fusion, which integrates visual and text
representation via coarse-grained prefix-guided interaction and fine-grained
correlation-aware fusion modules. We conduct extensive experiments to validate
that our MKGformer can obtain SOTA performance on four datasets of multimodal
link prediction, multimodal RE, and multimodal NER. Code is available in
this https URL.

本文提出了一种具有多级融合和混合变压器结构的 Multimodal Knowledge Graphs，用于解决多模态知识图谱完成任务中的问题，并在四个数据集上取得了 SOTA 性能。

多模态知识图谱完成的多层级融合混合变压器

Hybrid Transformer with Multi-level Fusion for Multimodal Knowledge Graph Completion

Most recently proposed approaches in dialogue state tracking (DST) leverage
the context and the last dialogue states to track current dialogue states,
which are often slot-value pairs. Although the context contains the complete
dialogue information, the information is usually indirect and even requires
reasoning to obtain. The information in the lastly predicted dialogue states is
direct, but when there is a prediction error, the dialogue information from
this source will be incomplete or erroneous. In this paper, we propose the
Dialogue State Tracking with Multi-Level Fusion of Predicted Dialogue States
and Conversations network (FPDSC). This model extracts information of each
dialogue turn by modeling interactions among each turn utterance, the
corresponding last dialogue states, and dialogue slots. Then the representation
of each dialogue turn is aggregated by a hierarchical structure to form the
passage information, which is utilized in the current turn of DST. Experimental
results validate the effectiveness of the fusion network with 55.03% and 59.07%
joint accuracy on MultiWOZ 2.0 and MultiWOZ 2.1 datasets, which reaches the
state-of-the-art performance. Furthermore, we conduct the deleted-value and
related-slot experiments on MultiWOZ 2.1 to evaluate our model.

本论文提出了一种多层次融合对话状态跟踪（DST）网络，通过建模每个对话轮次的交互，实现从以往对话历史和当前对话上下文中抽取对话状态信息，进行预测和融合，加强对当前对话状态的跟踪和理解，实验结果表明该模型在 MultiWOZ 2.0 和 MultiWOZ 2.1 数据集上取得了当前领先的 55.03% 和 59.07% 联合灵敏度值。