Real-time bidding (RTB) has become a major paradigm of display advertising.
Each ad impression generated from a user visit is auctioned in real time, where
demand-side platform (DSP) automatically provides bid price usually relying on
the ad impression value estimation and the optimal bid price determination.
However, the current bid strategy overlooks large randomness of the user
behaviors (e.g., click) and the cost uncertainty caused by the auction
competition. In this work, we explicitly factor in the uncertainty of estimated
ad impression values and model the risk preference of a DSP under a specific
state and market environment via a sequential decision process. Specifically,
we propose a novel adaptive risk-aware bidding algorithm with budget constraint
via reinforcement learning, which is the first to simultaneously consider
estimation uncertainty and the dynamic risk tendency of a DSP. We theoretically
unveil the intrinsic relation between the uncertainty and the risk tendency
based on value at risk (VaR). Consequently, we propose two instantiations to
model risk tendency, including an expert knowledge-based formulation embracing
three essential properties and an adaptive learning method based on
self-supervised reinforcement learning. We conduct extensive experiments on
public datasets and show that the proposed framework outperforms
state-of-the-art methods in practical settings.

本研究提出了一种基于强化学习的新型自适应风险感知竞价算法，旨在同时考虑估计不确定性和 DSP 的动态风险倾向，并通过公共数据集的广泛实验表明，该算法在实际 setting 中优于现有的最佳方法。