In the contemporary era of widespread online recruitment, resume
understanding has been widely acknowledged as a fundamental and crucial task,
which aims to extract structured information from resume documents
automatically. Compared to the traditional rule-based approaches, the
utilization of recently proposed pre-trained document understanding models can
greatly enhance the effectiveness of resume understanding. The present
approaches have, however, disregarded the hierarchical relations within the
structured information presented in resumes, and have difficulty parsing
resumes in an efficient manner. To this end, in this paper, we propose a novel
model, namely ERU, to achieve efficient resume understanding. Specifically, we
first introduce a layout-aware multi-modal fusion transformer for encoding the
segments in the resume with integrated textual, visual, and layout information.
Then, we design three self-supervised tasks to pre-train this module via a
large number of unlabeled resumes. Next, we fine-tune the model with a
multi-granularity sequence labeling task to extract structured information from
resumes. Finally, extensive experiments on a real-world dataset clearly
demonstrate the effectiveness of ERU.

本文提出了一种名为 ERU 的新型模型，通过引入多模态融合变压器编码简历段落的布局感知信息，设计三个自监督任务用于无标签简历的预训练，并通过多粒度序列标记任务对模型进行微调，从简历中提取结构化信息，实验证明 ERU 的有效性。

面向高效简历理解的多粒度多模态预训练方法

Towards Efficient Resume Understanding: A Multi-Granularity Multi-Modal  Pre-Training Approach

End-to-end autonomous driving has witnessed remarkable progress. However, the
extensive deployment of autonomous vehicles has yet to be realized, primarily
due to 1) inefficient multi-modal environment perception: how to integrate data
from multi-modal sensors more efficiently; 2) non-human-like scene
understanding: how to effectively locate and predict critical risky agents in
traffic scenarios like an experienced driver. To overcome these challenges, in
this paper, we propose a Multi-Modal fusion transformer incorporating Driver
Attention (M2DA) for autonomous driving. To better fuse multi-modal data and
achieve higher alignment between different modalities, a novel
Lidar-Vision-Attention-based Fusion (LVAFusion) module is proposed. By
incorporating driver attention, we empower the human-like scene understanding
ability to autonomous vehicles to identify crucial areas within complex
scenarios precisely and ensure safety. We conduct experiments on the CARLA
simulator and achieve state-of-the-art performance with less data in
closed-loop benchmarks. Source codes are available at
this https URL

提出了一种多模态融合变压器，结合驾驶员注意力用于自动驾驶，通过 Lidar-Vision-Attention-based Fusion 模块更好地融合多模态数据，并实现不同模态之间更高的对齐，同时结合驾驶员的注意力，赋予自动驾驶车辆类似于人类的场景理解能力，准确识别复杂场景中的关键区域，确保安全。在 CARLA 模拟器上进行实验，以较少的数据在闭环基准测试中取得了最先进的性能。

M2DA：多模态融合 Transformer 结合驾驶员注意力用于自动驾驶

M2DA: Multi-Modal Fusion Transformer Incorporating Driver Attention for  Autonomous Driving

Viewport prediction is a crucial aspect of tile-based 360 video streaming
system. However, existing trajectory based methods lack of robustness, also
oversimplify the process of information construction and fusion between
different modality inputs, leading to the error accumulation problem. In this
paper, we propose a tile classification based viewport prediction method with
Multi-modal Fusion Transformer, namely MFTR. Specifically, MFTR utilizes
transformer-based networks to extract the long-range dependencies within each
modality, then mine intra- and inter-modality relations to capture the combined
impact of user historical inputs and video contents on future viewport
selection. In addition, MFTR categorizes future tiles into two categories: user
interested or not, and selects future viewport as the region that contains most
user interested tiles. Comparing with predicting head trajectories, choosing
future viewport based on tile's binary classification results exhibits better
robustness and interpretability. To evaluate our proposed MFTR, we conduct
extensive experiments on two widely used PVS-HM and Xu-Gaze dataset. MFTR shows
superior performance over state-of-the-art methods in terms of average
prediction accuracy and overlap ratio, also presents competitive computation
efficiency.

提出了一种基于瓷砖分类的视口预测方法，使用多模态融合变压器提取每种模态的长距离依赖关系，从而捕捉用户历史输入和视频内容对未来视口选择的综合影响，并根据瓷砖的二分类结果选择未来视口，具有更好的鲁棒性和解释性。在两个常用的 PVS-HM 和 Xu-Gaze 数据集上进行了广泛实验，MFTR 表现出卓越的性能，包括平均预测准确率和重叠率，同时展现出有竞争力的计算效率。