Real-time bidding is the new paradigm of programmatic advertising. An
advertiser wants to make the intelligent choice of utilizing a
\textbf{Demand-Side Platform} to improve the performance of their ad campaigns.
Existing approaches are struggling to provide a satisfactory solution for
bidding optimization due to stochastic bidding behavior. In this paper, we
proposed a multi-agent reinforcement learning architecture for RTB with
functional optimization. We designed four agents bidding environment: three
Lagrange-multiplier based functional optimization agents and one baseline agent
(without any attribute of functional optimization) First, numerous attributes
have been assigned to each agent, including biased or unbiased win probability,
Lagrange multiplier, and click-through rate. In order to evaluate the proposed
RTB strategy's performance, we demonstrate the results on ten sequential
simulated auction campaigns. The results show that agents with functional
actions and rewards had the most significant average winning rate and winning
surplus, given biased and unbiased winning information respectively. The
experimental evaluations show that our approach significantly improve the
campaign's efficacy and profitability.

本文提出了一种多智能体强化学习架构以用于实时竞价广告，使用三个 Lagrange 乘数基于功能优化以及一个基准代理程序进行竞标环境设计。实验结果表明，相比其他代理程序，具有功能优化的代理程序在广告竞标过程中获胜速率和盈余率上有显著提升，表现出优异的表现和盈利性。