While there has been remarkable progress recently in the fields of
manipulation and locomotion, mobile manipulation remains a long-standing
challenge. Compared to locomotion or static manipulation, a mobile system must
make a diverse range of long-horizon tasks feasible in unstructured and dynamic
environments. While the applications are broad and interesting, there are a
plethora of challenges in developing these systems such as coordination between
the base and arm, reliance on onboard perception for perceiving and interacting
with the environment, and most importantly, simultaneously integrating all
these parts together. Prior works approach the problem using disentangled
modular skills for mobility and manipulation that are trivially tied together.
This causes several limitations such as compounding errors, delays in
decision-making, and no whole-body coordination. In this work, we present a
reactive mobile manipulation framework that uses an active visual system to
consciously perceive and react to its environment. Similar to how humans
leverage whole-body and hand-eye coordination, we develop a mobile manipulator
that exploits its ability to move and see, more specifically -- to move in
order to see and to see in order to move. This allows it to not only move
around and interact with its environment but also, choose "when" to perceive
"what" using an active visual system. We observe that such an agent learns to
navigate around complex cluttered scenarios while displaying agile whole-body
coordination using only ego-vision without needing to create environment maps.
Results visualizations and videos at this https URL

通过使用主动视觉系统来有意识地感知和应对环境，我们开发了一种移动操纵框架，该框架利用移动机械手的能力来移动和观察，从而能够在复杂混乱的场景中导航并展示灵活的全身协调，而无需创建环境地图。

SPIN: 同时感知、交互与导航

SPIN: Simultaneous Perception, Interaction and Navigation

Analysis and interpretation of egocentric video data is becoming more and
more important with the increasing availability and use of wearable cameras.
Exploring and fully understanding affinities and differences between ego and
allo (or third-person) vision is paramount for the design of effective methods
to process, analyse and interpret egocentric data. In addition, a deeper
understanding of ego-vision and its peculiarities may enable new research
perspectives in which first person viewpoints can act either as a mean for
easily acquiring large amounts of data to be employed in general-purpose
recognition systems, and as a challenging test-bed to assess the usability of
techniques specifically tailored to deal with allocentric vision on more
challenging settings. Our work, with an eye to cognitive science findings,
leverages transfer learning in Convolutional Neural Networks to demonstrate
capabilities and limitations of an implicitly learnt view-invariant
representation in the specific case of action recognition.

本文主要研究如何利用佩戴式摄像头所获得的数据进行行为识别。研究发现，使用卷积神经网络进行自我视角识别可以开辟行为识别领域的新研究视角。