With the end of Moore's Law and the increasing demand for computing, photonic
accelerators are garnering considerable attention. This is due to the physical
characteristics of light, such as high bandwidth and multiplicity, and the
various synchronization phenomena that emerge in the realm of laser physics.
These factors come into play as computer performance approaches its limits. In
this study, we explore the application of a laser network, acting as a photonic
accelerator, to the competitive multi-armed bandit problem. In this context,
conflict avoidance is key to maximizing environmental rewards. We
experimentally demonstrate cooperative decision-making using zero-lag and lag
synchronization within a network of four semiconductor lasers. Lag
synchronization of chaos realizes effective decision-making and zero-delay
synchronization is responsible for the realization of the collision avoidance
function. We experimentally verified a low collision rate and high reward in a
fundamental 2-player, 2-slot scenario, and showed the scalability of this
system. This system architecture opens up new possibilities for intelligent
functionalities in laser dynamics.

通过实验验证光网络作为光子加速器在竞争型多臂赌博问题中的应用，实现协同决策、冲突避免、低碰撞率和高奖励，并展示了该系统的可扩展性以及在激光动力学领域中智能功能的新可能性。

激光网络中通过滞后和零滞后同步实现无冲突的联合决策

Conflict-free joint decision by lag and zero-lag synchronization in  laser network

In a multi-agent environment, In order to overcome and alleviate the
non-stationarity of the multi-agent environment, the mainstream method is to
adopt the framework of Centralized Training Decentralized Execution (CTDE).
This thesis is based on the framework of CTDE, and studies the cooperative
decision-making of multi-agent based on the Multi-Agent Proximal Policy
Optimization (MAPPO) algorithm for multi-agent proximal policy optimization. In
order to alleviate the non-stationarity of the multi-agent environment, a
multi-agent communication mechanism based on weight scheduling and attention
module is introduced. Different agents can alleviate the non-stationarity
caused by local observations through information exchange between agents,
assisting in the collaborative decision-making of agents. The specific method
is to introduce a communication module in the policy network part. The
communication module is composed of a weight generator, a weight scheduler, a
message encoder, a message pool and an attention module. Among them, the weight
generator and weight scheduler will generate weights as the selection basis for
communication, the message encoder is used to compress and encode communication
information, the message pool is used to store communication messages, and the
attention module realizes the interactive processing of the agent's own
information and communication information. This thesis proposes a Multi-Agent
Communication and Global Information Optimization Proximal Policy
Optimization(MCGOPPO)algorithm, and conducted experiments in the SMAC and the
MPE. The experimental results show that the improvement has achieved certain
effects, which can better alleviate the non-stationarity of the multi-agent
environment, and improve the collaborative decision-making ability among the
agents.

本篇论文基于 CTDE 框架，研究基于 MAPPO 算法的多智能体合作决策，并引入了基于权重调度和注意力机制的多智能体通信机制以缓解多智能体环境中的非稳定性。提出了 MCGOPPO 算法，并在 SMAC 和 MPE 上进行了实验，结果表明此算法可以改善多智能体环境中的非稳定性，提高多智能体间的协同决策能力。

基于深度强化学习的多智能体通信与协作决策研究

Research on Multi-Agent Communication and Collaborative Decision-Making  Based on Deep Reinforcement Learning

The multi-vehicle pursuit (MVP), as a problem abstracted from various
real-world scenarios, is becoming a hot research topic in Intelligent
Transportation System (ITS). The combination of Artificial Intelligence (AI)
and connected vehicles has greatly promoted the research development of MVP.
However, existing works on MVP pay little attention to the importance of
information exchange and cooperation among pursuing vehicles under the complex
urban traffic environment. This paper proposed a graded-Q reinforcement
learning with information-enhanced state encoder (GQRL-IESE) framework to
address this hierarchical collaborative multi-vehicle pursuit (HCMVP) problem.
In the GQRL-IESE, a cooperative graded Q scheme is proposed to facilitate the
decision-making of pursuing vehicles to improve pursuing efficiency. Each
pursuing vehicle further uses a deep Q network (DQN) to make decisions based on
its encoded state. A coordinated Q optimizing network adjusts the individual
decisions based on the current environment traffic information to obtain the
global optimal action set. In addition, an information-enhanced state encoder
is designed to extract critical information from multiple perspectives and uses
the attention mechanism to assist each pursuing vehicle in effectively
determining the target. Extensive experimental results based on SUMO indicate
that the total timestep of the proposed GQRL-IESE is less than other methods on
average by 47.64%, which demonstrates the excellent pursuing efficiency of the
GQRL-IESE. Codes are outsourced in this https URL.

本文提出了一种基于分级协同增强的强化学习框架（GQRL-IESE）来解决复杂的城市交通环境下的分层多车追逐问题，实验证明该框架的效率比其他方法平均快了 47.64％以上。