Task execution for object rearrangement could be challenged by Task-Level Perturbations (TLP), i.e., unexpected object additions, removals, and displacements that can disrupt underlying visual policies and fundamentally compromise task feasibility and progress. To address these challenges, we present LangPert, a language-based framework designed to detect and mitigate TLP situations in tabletop rearrangement tasks. LangPert integrates a Visual Language Model (VLM) to comprehensively monitor policy's skill execution and environmental TLP, while leveraging the Hierarchical Chain-of-Thought (HCoT) reasoning mechanism to enhance the Large Language Model (LLM)'s contextual understanding and generate adaptive, corrective skill-execution plans. Our experimental results demonstrate that LangPert handles diverse TLP situations more effectively than baseline methods, achieving higher task completion rates, improved execution efficiency, and potential generalization to unseen scenarios.

本研究解决了任务执行过程中物体重排的任务级扰动（TLP）问题，如意外的物体增加、移除和位移，这些扰动会破坏视觉策略并影响任务的可行性和进展。论文提出的LangPert框架采用视觉语言模型（VLM）和层次思维链（HCoT）推理机制，有效检测和缓解TLP，提高了任务完成率和执行效率，显示出优秀的适应能力和潜在的广泛应用。

LangPert：检测和处理任务级扰动以增强物体重排的鲁棒性