This work views the multi-agent system and its surrounding environment as a
co-evolving system, where the behavior of one affects the other. The goal is to
take both agent actions and environment configurations as decision variables,
and optimize these two components in a coordinated manner to improve some
measure of interest. Towards this end, we consider the problem of decentralized
multi-agent navigation in cluttered environments. By introducing two
sub-objectives of multi-agent navigation and environment optimization, we
propose an $\textit{agent-environment co-optimization}$ problem and develop a
$\textit{coordinated algorithm}$ that alternates between these sub-objectives
to search for an optimal synthesis of agent actions and obstacle configurations
in the environment; ultimately, improving the navigation performance. Due to
the challenge of explicitly modeling the relation between agents, environment
and performance, we leverage policy gradient to formulate a model-free learning
mechanism within the coordinated framework. A formal convergence analysis shows
that our coordinated algorithm tracks the local minimum trajectory of an
associated time-varying non-convex optimization problem. Extensive numerical
results corroborate theoretical findings and show the benefits of
co-optimization over baselines. Interestingly, the results also indicate that
optimized environment configurations are able to offer structural guidance that
is key to de-conflicting agents in motion.

通过引入多智能体系统和环境协同演化的观点，本研究提出了一个多智能体导航的去中心化问题，通过协同算法交替优化智能体行为和环境配置，选择最优的智能体动作和障碍物配置，以提高导航性能。通过策略梯度方法，在协同框架中建立了无模型学习机制，并进行了收敛性分析和对比实验，结果显示优化的环境配置对解决智能体运动冲突至关重要。