Embodied AI models often employ off the shelf vision backbones like CLIP to
encode their visual observations. Although such general purpose representations
encode rich syntactic and semantic information about the scene, much of this
information is often irrelevant to the specific task at hand. This introduces
noise within the learning process and distracts the agent's focus from
task-relevant visual cues. Inspired by selective attention in humans-the
process through which people filter their perception based on their
experiences, knowledge, and the task at hand-we introduce a parameter-efficient
approach to filter visual stimuli for embodied AI. Our approach induces a
task-conditioned bottleneck using a small learnable codebook module. This
codebook is trained jointly to optimize task reward and acts as a
task-conditioned selective filter over the visual observation. Our experiments
showcase state-of-the-art performance for object goal navigation and object
displacement across 5 benchmarks, ProcTHOR, ArchitecTHOR, RoboTHOR, AI2-iTHOR,
and ManipulaTHOR. The filtered representations produced by the codebook are
also able generalize better and converge faster when adapted to other
simulation environments such as Habitat. Our qualitative analyses show that
agents explore their environments more effectively and their representations
retain task-relevant information like target object recognition while ignoring
superfluous information about other objects. Code and pretrained models are
available at our project website: this https URL

基于选择性注意力的体感人工智能的参数高效的视觉刺激过滤方法在目标导航和目标位置变换等任务中展现出最先进的性能，并且在其他仿真环境中具有更好的泛化性和更快的收敛速度。