Order execution is a fundamental task in quantitative finance, aiming at
finishing acquisition or liquidation for a number of trading orders of the
specific assets. Recent advance in model-free reinforcement learning (RL)
provides a data-driven solution to the order execution problem. However, the
existing works always optimize execution for an individual order, overlooking
the practice that multiple orders are specified to execute simultaneously,
resulting in suboptimality and bias. In this paper, we first present a
multi-agent RL (MARL) method for multi-order execution considering practical
constraints. Specifically, we treat every agent as an individual operator to
trade one specific order, while keeping communicating with each other and
collaborating for maximizing the overall profits. Nevertheless, the existing
MARL algorithms often incorporate communication among agents by exchanging only
the information of their partial observations, which is inefficient in
complicated financial market. To improve collaboration, we then propose a
learnable multi-round communication protocol, for the agents communicating the
intended actions with each other and refining accordingly. It is optimized
through a novel action value attribution method which is provably consistent
with the original learning objective yet more efficient. The experiments on the
data from two real-world markets have illustrated superior performance with
significantly better collaboration effectiveness achieved by our method.

在这篇论文中，我们介绍了一种多智能体强化学习方法来同时执行多个交易订单，并通过可学习的多轮通信协议改善合作效果。通过使用模型无关的强化学习方法，我们提供了一种解决交易执行问题的数据驱动方法。实验证明，我们的方法在两个真实市场的数据上表现出优越的性能和显著改善的合作效果。

学习多智能体意图感知通信以实现金融业务的最佳多级执行

Learning Multi-Agent Intention-Aware Communication for Optimal  Multi-Order Execution in Finance

As a fundamental problem in algorithmic trading, order execution aims at
fulfilling a specific trading order, either liquidation or acquirement, for a
given instrument. Towards effective execution strategy, recent years have
witnessed the shift from the analytical view with model-based market
assumptions to model-free perspective, i.e., reinforcement learning, due to its
nature of sequential decision optimization. However, the noisy and yet
imperfect market information that can be leveraged by the policy has made it
quite challenging to build up sample efficient reinforcement learning methods
to achieve effective order execution. In this paper, we propose a novel
universal trading policy optimization framework to bridge the gap between the
noisy yet imperfect market states and the optimal action sequences for order
execution. Particularly, this framework leverages a policy distillation method
that can better guide the learning of the common policy towards practically
optimal execution by an oracle teacher with perfect information to approximate
the optimal trading strategy. The extensive experiments have shown significant
improvements of our method over various strong baselines, with reasonable
trading actions.

该研究提出了一种基于增强学习和策略优化的框架，通过引入一个策略蒸馏方法来优化订单执行的普遍交易策略，以更好地指导基于模型无关的方法实现有效的订单执行。