In the vision and language navigation task, the agent may encounter ambiguous
situations that are hard to interpret by just relying on visual information and
natural language instructions. We propose an interactive learning framework to
endow the agent with the ability to ask for users' help in such situations. As
part of this framework, we investigate multiple learning approaches for the
agent with different levels of complexity. The simplest model-confusion-based
method lets the agent ask questions based on its confusion, relying on the
predefined confidence threshold of a next action prediction model. To build on
this confusion-based method, the agent is expected to demonstrate more
sophisticated reasoning such that it discovers the timing and locations to
interact with a human. We achieve this goal using reinforcement learning (RL)
with a proposed reward shaping term, which enables the agent to ask questions
only when necessary. The success rate can be boosted by at least 15% with only
one question asked on average during the navigation. Furthermore, we show that
the RL agent is capable of adjusting dynamically to noisy human responses.
Finally, we design a continual learning strategy, which can be viewed as a data
augmentation method, for the agent to improve further utilizing its interaction
history with a human. We demonstrate the proposed strategy is substantially
more realistic and data-efficient compared to previously proposed
pre-exploration techniques.

本文提出了一个交互式学习框架以及多种学习方法应对视觉和语言导航任务中出现的模糊情况，引入强化学习并设计奖励塑形项使智能体仅在必要时提问，同时展示了持续学习策略的有效性。

Just Ask: 视觉与语言导航的交互学习框架

Just Ask:An Interactive Learning Framework for Vision and Language  Navigation

Various domain users are increasingly leveraging real-time social media data
to gain rapid situational awareness. However, due to the high noise in the
deluge of data, effectively determining semantically relevant information can
be difficult, further complicated by the changing definition of relevancy by
each end user for different events. The majority of existing methods for short
text relevance classification fail to incorporate users' knowledge into the
classification process. Existing methods that incorporate interactive user
feedback focus on historical datasets. Therefore, classifiers cannot be
interactively retrained for specific events or user-dependent needs in
real-time. This limits real-time situational awareness, as streaming data that
is incorrectly classified cannot be corrected immediately, permitting the
possibility for important incoming data to be incorrectly classified as well.
We present a novel interactive learning framework to improve the classification
process in which the user iteratively corrects the relevancy of tweets in
real-time to train the classification model on-the-fly for immediate predictive
improvements. We computationally evaluate our classification model adapted to
learn at interactive rates. Our results show that our approach outperforms
state-of-the-art machine learning models. In addition, we integrate our
framework with the extended Social Media Analytics and Reporting Toolkit
(SMART) 2.0 system, allowing the use of our interactive learning framework
within a visual analytics system tailored for real-time situational awareness.
To demonstrate our framework's effectiveness, we provide domain expert feedback
from first responders who used the extended SMART 2.0 system.

提出了一种新颖的交互式学习框架来提高对实时推文的分类过程，该框架允许用户在实时学习的基础上迭代地纠正推文的相关性以训练分类模型，实验结果表明该方法优于现有的机器学习模型，结合 SMART 系统，可用于实时情境感知并满足不同用户依赖需求。