With the recent prevalence of Reinforcement Learning (RL), there have been
tremendous interests in utilizing RL for online advertising in recommendation
platforms (e.g., e-commerce and news feed sites). However, most RL-based
advertising algorithms focus on optimizing ads' revenue while ignoring the
possible negative influence of ads on user experience of recommended items
(products, articles and videos). Developing an optimal advertising algorithm in
recommendations faces immense challenges because interpolating ads improperly
or too frequently may decrease user experience, while interpolating fewer ads
will reduce the advertising revenue. Thus, in this paper, we propose a novel
advertising strategy for the rec/ads trade-off. To be specific, we develop an
RL-based framework that can continuously update its advertising strategies and
maximize reward in the long run. Given a recommendation list, we design a novel
Deep Q-network architecture that can determine three internally related tasks
jointly, i.e., (i) whether to interpolate an ad or not in the recommendation
list, and if yes, (ii) the optimal ad and (iii) the optimal location to
interpolate. The experimental results based on real-world data demonstrate the
effectiveness of the proposed framework.

本文提出了一种基于强化学习的广告策略，通过设计一个新型的深度 Q 网络结构，可以连续更新其广告策略，在推荐列表中插入广告或不插入广告，并选择最优广告和最佳插入位置，使得推荐广告效果显著提升。