Recent advancements in Generative Artificial Intelligence, particularly in
the realm of Large Language Models (LLMs) and Large Vision Language Models
(LVLMs), have enabled the prospect of leveraging cognitive planners within
robotic systems. This work focuses on solving the object goal navigation
problem by mimicking human cognition to attend, perceive and store task
specific information and generate plans with the same. We introduce a
comprehensive framework capable of exploring an unfamiliar environment in
search of an object by leveraging the capabilities of Large Language
Models(LLMs) and Large Vision Language Models (LVLMs) in understanding the
underlying semantics of our world. A challenging task in using LLMs to generate
high level sub-goals is to efficiently represent the environment around the
robot. We propose to use a 3D scene modular representation, with semantically
rich descriptions of the object, to provide the LLM with task relevant
information. But providing the LLM with a mass of contextual information (rich
3D scene semantic representation), can lead to redundant and inefficient plans.
We propose to use an LLM based pruner that leverages the capabilities of
in-context learning to prune out irrelevant goal specific information.

利用大型语言模型（LLMs）和大型视觉语言模型（LVLMs）的能力，本研究提出了一个综合框架，模仿人类认知以实现对象目标导航问题的解决，通过关注、感知和存储任务特定信息，并生成相应计划。为了有效表示机器人周围的环境，提出使用语义丰富的 3D 场景模块化表示，并引入基于 LLM 的修剪策略以消除无关的目标特定信息。