While Large Language Models (LLMs) have demonstrated impressive
accomplishments in both reasoning and planning, their abilities in multi-agent
collaborations remains largely unexplored. This study evaluates LLM-based
agents in a multi-agent cooperative text game with Theory of Mind (ToM)
inference tasks, comparing their performance with Multi-Agent Reinforcement
Learning (MARL) and planning-based baselines. We observed evidence of emergent
collaborative behaviors and high-order Theory of Mind capabilities among
LLM-based agents. Our results reveal limitations in LLM-based agents' planning
optimization due to systematic failures in managing long-horizon contexts and
hallucination about the task state. We explore the use of explicit belief state
representations to mitigate these issues, finding that it enhances task
performance and the accuracy of ToM inferences for LLM-based agents.

评估了基于大型语言模型的多智能体协作文本游戏中的理论推理任务，并与多智能体强化学习和基于计划的基准进行比较。研究发现基于大型语言模型的智能体表现出紧密合作行为和高级理论推理能力，但也存在在管理长期视野环境和任务状态幻觉方面的规划优化限制。通过利用显式信念状态表示来减轻这些问题，改善了基于大型语言模型的智能体的任务表现和理论推理准确性。

通过大型语言模型实现多智能体协作的心智理论

Theory of Mind for Multi-Agent Collaboration via Large Language Models

Camera-only 3D detection provides an economical solution with a simple
configuration for localizing objects in 3D space compared to LiDAR-based
detection systems. However, a major challenge lies in precise depth estimation
due to the lack of direct 3D measurements in the input. Many previous methods
attempt to improve depth estimation through network designs, e.g., deformable
layers and larger receptive fields. This work proposes an orthogonal direction,
improving the camera-only 3D detection by introducing multi-agent
collaborations. Our proposed collaborative camera-only 3D detection (CoCa3D)
enables agents to share complementary information with each other through
communication. Meanwhile, we optimize communication efficiency by selecting the
most informative cues. The shared messages from multiple viewpoints
disambiguate the single-agent estimated depth and complement the occluded and
long-range regions in the single-agent view. We evaluate CoCa3D in one
real-world dataset and two new simulation datasets. Results show that CoCa3D
improves previous SOTA performances by 44.21% on DAIR-V2X, 30.60% on OPV2V+,
12.59% on CoPerception-UAVs+ for AP@70. Our preliminary results show a
potential that with sufficient collaboration, the camera might overtake LiDAR
in some practical scenarios. We released the dataset and code at
this https URL and
this https URL

本文提出了一种通过引入多智能体协作来提高仅使用摄像机的 3D 检测的方法，称为 CoCa3D。实验表明，CoCa3D 在真实数据集和两个新的模拟数据集上的表现优于之前最好的方法，并显示出摄像机在某些实际场景中可能会超越 LiDAR 的潜力。

合作帮助相机在三维检测中超越激光雷达

Collaboration Helps Camera Overtake LiDAR in 3D Detection

Understanding and interpreting human actions is a long-standing challenge and
a critical indicator of perception in artificial intelligence. However, a few
imperative components of daily human activities are largely missed in prior
literature, including the goal-directed actions, concurrent multi-tasks, and
collaborations among multi-agents. We introduce the LEMMA dataset to provide a
single home to address these missing dimensions with meticulously designed
settings, wherein the number of tasks and agents varies to highlight different
learning objectives. We densely annotate the atomic-actions with human-object
interactions to provide ground-truths of the compositionality, scheduling, and
assignment of daily activities. We further devise challenging compositional
action recognition and action/task anticipation benchmarks with baseline models
to measure the capability of compositional action understanding and temporal
reasoning. We hope this effort would drive the machine vision community to
examine goal-directed human activities and further study the task scheduling
and assignment in the real world.

介绍了 LEMA 数据集，通过密集注释原子动作和人 - 物互动，为日常活动的组成性，调度性和分配性提供了地面真实性，进一步设计了具有基线模型的具有挑战性的组成动作识别和动作 / 任务预测基准，旨在推动机器视觉社区研究真实世界中的人类目标导向活动和任务调度和分配。