Recently, there has been extensive study of cooperative multi-agent
multi-armed bandits where a set of distributed agents cooperatively play the
same multi-armed bandit game. The goal is to develop bandit algorithms with the
optimal group and individual regrets and low communication between agents. The
prior work tackled this problem using two paradigms: leader-follower and fully
distributed algorithms. Prior algorithms in both paradigms achieve the optimal
group regret. The leader-follower algorithms achieve constant communication
costs but fail to achieve optimal individual regrets. The state-of-the-art
fully distributed algorithms achieve optimal individual regrets but fail to
achieve constant communication costs. This paper presents a simple yet
effective communication policy and integrates it into a learning algorithm for
cooperative bandits. Our algorithm achieves the best of both paradigms: optimal
individual regret and constant communication costs.

合作多智能体多臂赌博算法中的通信策略，既实现了最优个体遗憾，又具有恒定的通信成本。

合作多智能体赌博机：优化个体懊悔并具有恒定通讯开销的分布式算法

Cooperative Multi-agent Bandits: Distributed Algorithms with Optimal  Individual Regret and Constant Communication Costs

The Bilevel Optimization Problem is a hierarchical optimization problem with
two agents, a leader and a follower. The leader make their own decisions first,
and the followers make the best choices accordingly. The leader knows the
information of the followers, and the goal of the problem is to find the
optimal solution by considering the reactions of the followers from the
leader's point of view. For the Bilevel Optimization Problem, there are no
general and efficient algorithms or commercial solvers to get an optimal
solution, and it is very difficult to get a good solution even for a simple
problem. In this paper, we propose a deep learning approach using Graph Neural
Networks to solve the bilevel knapsack problem. We train the model to predict
the leader's solution and use it to transform the hierarchical optimization
problem into a single-level optimization problem to get the solution. Our model
found the feasible solution that was about 500 times faster than the exact
algorithm with $1.7\%$ optimal gap. Also, our model performed well on problems
of different size from the size it was trained on.

本文提出了使用图神经网络解决双层背包问题的深度学习方法，该方法通过训练模型来预测领导者的解决方案，从而将层次优化问题转化为单层优化问题，我们的模型找到的可行解比精确算法快了约 500 倍，并且在训练大小不同的问题上表现良好。