E-commerce platforms usually display a mixed list of ads and organic items in
feed. One key problem is to allocate the limited slots in the feed to maximize
the overall revenue as well as improve user experience, which requires a good
model for user preference. Instead of modeling the influence of individual
items on user behaviors, the arrangement signal models the influence of the
arrangement of items and may lead to a better allocation strategy. However,
most of previous strategies fail to model such a signal and therefore result in
suboptimal performance. In addition, the percentage of ads exposed (PAE) is an
important indicator in ads allocation. Excessive PAE hurts user experience
while too low PAE reduces platform revenue. Therefore, how to constrain the PAE
within a certain range while keeping personalized recommendation under the PAE
constraint is a challenge. In this paper, we propose Cross Deep Q Network
(Cross DQN) to extract the crucial arrangement signal by crossing the
embeddings of different items and modeling the crossed sequence by
multi-channel attention. Besides, we propose an auxiliary loss for batch-level
constraint on PAE to tackle the above-mentioned challenge. Our model results in
higher revenue and better user experience than state-of-the-art baselines in
offline experiments. Moreover, our model demonstrates a significant improvement
in the online A/B test and has been fully deployed on Meituan feed to serve
more than 300 millions of customers.

本文提出了 Cross Deep Q Network（Cross DQN）的模型，通过交叉不同物品的嵌入来提取重要的排列信号并通过多通道注意力建模。此外，我们提出了一种辅助损失来处理广告暴露率的批级约束，以在保持个性化推荐的同时将广告暴露率限制在一定范围内，该模型经离线和在线实验证明在平台上获得了更高的收入和更好的用户体验。