Autonomous systems deployed on edge devices face significant challenges, including resource constraints, real-time processing demands, and adapting to dynamic environments. This work introduces ATLASv2, a novel system that integrates a fine-tuned TinyLLM, real-time object detection, and efficient path planning to enable hierarchical, multi-task navigation and manipulation all on the edge device, Jetson Nano. ATLASv2 dynamically expands its navigable landmarks by detecting and localizing objects in the environment which are saved to its internal knowledge base to be used for future task execution. We evaluate ATLASv2 in real-world environments, including a handcrafted home and office setting constructed with diverse objects and landmarks. Results show that ATLASv2 effectively interprets natural language instructions, decomposes them into low-level actions, and executes tasks with high success rates. By leveraging generative AI in a fully on-board framework, ATLASv2 achieves optimized resource utilization with minimal prompting latency and power consumption, bridging the gap between simulated environments and real-world applications.

本研究针对边缘设备上的自主系统面临的资源限制和实时处理要求，提出了ATLASv2系统。该系统结合了微调的TinyLLM、实时物体检测和高效路径规划，实现了多任务导航和操作，且在真实环境中测试显示出高成功率，使得生成式AI在边缘设备上的应用变得更加高效。

ATLASv2：基于大型语言模型的自适应地标获取与边缘导航