Vision-and-Language Navigation (VLN) aims to develop intelligent agents to
navigate in unseen environments only through language and vision supervision.
In the recently proposed continuous settings (continuous VLN), the agent must
act in a free 3D space and faces tougher challenges like real-time execution,
complex instruction understanding, and long action sequence prediction. For a
better performance in continuous VLN, we design a multi-level instruction
understanding procedure and propose a novel model, Multi-Level Attention
Network (MLANet). The first step of MLANet is to generate sub-instructions
efficiently. We design a Fast Sub-instruction Algorithm (FSA) to segment the
raw instruction into sub-instructions and generate a new sub-instruction
dataset named ``FSASub". FSA is annotation-free and faster than the current
method by 70 times, thus fitting the real-time requirement in continuous VLN.
To solve the complex instruction understanding problem, MLANet needs a global
perception of the instruction and observations. We propose a Multi-Level
Attention (MLA) module to fuse vision, low-level semantics, and high-level
semantics, which produce features containing a dynamic and global comprehension
of the task. MLA also mitigates the adverse effects of noise words, thus
ensuring a robust understanding of the instruction. To correctly predict
actions in long trajectories, MLANet needs to focus on what sub-instruction is
being executed every step. We propose a Peak Attention Loss (PAL) to improve
the flexible and adaptive selection of the current sub-instruction. PAL
benefits the navigation agent by concentrating its attention on the local
information, thus helping the agent predict the most appropriate actions. We
train and test MLANet in the standard benchmark. Experiment results show MLANet
outperforms baselines by a significant margin.

为了更好地在连续的虚拟现实环境中实现语音导航，本文提出了一个多层次的指令理解机制和一个名为 MLANet 的新模型，它包含子指令生成的快速算法 (FSA)、多层次关注模块 (MLA) 来调和指令和视觉信号，以及用于提高对当前子指令选择能力的峰值关注损失 (PAL)，实验结果表明 MLANet 优于基线模型。

MLANet：基于子指令的多级注意力网络，用于连续的视觉语言导航

MLANet: Multi-Level Attention Network with Sub-instruction for Continuous Vision-and-Language Navigation

This paper proposes an attribute-guided multi-level attention network
(AG-MLAN) to learn fine-grained fashion similarity. AG-MLAN is able to make a
more accurate attribute positioning and capture more discriminative features
under the guidance of the specified attribute. Specifically, the AG-MLAN
contains two branches, branch 1 aims to force the model to recognize different
attributes, while branch 2 aims to learn multiple attribute-specific embedding
spaces for measuring the fine-grained similarity. We first improve the
Convolutional Neural Network (CNN) backbone to extract hierarchical feature
representations, then the extracted feature representations are passed into
branch 1 for attribute classification and branch 2 for multi-level feature
extraction. In branch 2, we first propose a multi-level attention module to
extract a more discriminative representation under the guidance of a specific
attribute. Then, we adopt a masked embedding module to learn attribute-aware
embedding. Finally, the AG-MLAN is trained with a weighted loss of the
classification loss in branch 1 and the triplet loss of the masked embedding
features in branch 2 to further improve the accuracy in attribute location.
Extensive experiments on the DeepFashion, FashionAI, and Zappos50k datasets
show the effectiveness of AG-MLAN for fine-grained fashion similarity learning
and its potential for attribute-guided retrieval tasks. The proposed AG-MLAN
outperforms the state-of-the-art methods in the fine-grained fashion similarity
retrieval task.

本研究提出了一种属性引导的多层注意力网络（AG-MLAN），能够在指定属性的引导下更准确地定位属性位置和捕捉区分性特征，并通过对深度时尚、FashionAI 和 Zappos50k 数据集上的实验验证了其在细粒度时尚相似度学习和属性引导检索任务中的有效性。该模型在细粒度时尚相似度检索任务中的表现超过了现有最优方法。

细粒度时尚检索的属性引导多级注意力网络

Attribute-Guided Multi-Level Attention Network for Fine-Grained Fashion Retrieval

Social recommendation has emerged to leverage social connections among users
for predicting users' unknown preferences, which could alleviate the data
sparsity issue in collaborative filtering based recommendation. Early
approaches relied on utilizing each user's first-order social neighbors'
interests for better user modeling and failed to model the social influence
diffusion process from the global social network structure. Recently, we
propose a preliminary work of a neural influence diffusion network (i.e.,
DiffNet) for social recommendation (Diffnet), which models the recursive social
diffusion process to capture the higher-order relationships for each user.
However, we argue that, as users play a central role in both user-user social
network and user-item interest network, only modeling the influence diffusion
process in the social network would neglect the users' latent collaborative
interests in the user-item interest network. In this paper, we propose
DiffNet++, an improved algorithm of DiffNet that models the neural influence
diffusion and interest diffusion in a unified framework. By reformulating the
social recommendation as a heterogeneous graph with social network and interest
network as input, DiffNet++ advances DiffNet by injecting these two network
information for user embedding learning at the same time. This is achieved by
iteratively aggregating each user's embedding from three aspects: the user's
previous embedding, the influence aggregation of social neighbors from the
social network, and the interest aggregation of item neighbors from the
user-item interest network. Furthermore, we design a multi-level attention
network that learns how to attentively aggregate user embeddings from these
three aspects. Finally, extensive experimental results on two real-world
datasets clearly show the effectiveness of our proposed model.

本文提出了 DiffNet++ 算法，结合社交网络与兴趣网络，通过三个方面迭代聚合用户嵌入，设计了多级注意力网络，有效地解决了社交推荐中数据稀疏问题。