Modern information retrieval systems, including web search, ads placement,
and recommender systems, typically rely on learning from user feedback. Click
models, which study how users interact with a ranked list of items, provide a
useful understanding of user feedback for learning ranking models. Constructing
"right" dependencies is the key of any successful click model. However,
probabilistic graphical models (PGMs) have to rely on manually assigned
dependencies, and oversimplify user behaviors. Existing neural network based
methods promote PGMs by enhancing the expressive ability and allowing flexible
dependencies, but still suffer from exposure bias and inferior estimation. In
this paper, we propose a novel framework, Adversarial Imitation Click Model
(AICM), based on imitation learning. Firstly, we explicitly learn the reward
function that recovers users' intrinsic utility and underlying intentions.
Secondly, we model user interactions with a ranked list as a dynamic system
instead of one-step click prediction, alleviating the exposure bias problem.
Finally, we minimize the JS divergence through adversarial training and learn a
stable distribution of click sequences, which makes AICM generalize well across
different distributions of ranked lists. A theoretical analysis has indicated
that AICM reduces the exposure bias from $O(T^2)$ to $O(T)$. Our studies on a
public web search dataset show that AICM not only outperforms state-of-the-art
models in traditional click metrics but also achieves superior performance in
addressing the exposure bias and recovering the underlying patterns of click
sequences.

本文提出了一种基于模仿学习的对抗学习点击模型框架（Adversarial Imitation Click Model, AICM），通过明确地学习恢复用户的内在效用和潜在意图的奖励函数，将用户与排名列表的交互建模为动态系统（而不是单步点击预测），通过对抗训练最小化 JS 发散来学习稳定的点击序列分布，从而将暴露偏差从 $O (T^2)$ 降到了 $O (T)$，结果表明 AICM 在传统点击度量和去暴露方面具有优异的性能。

一种面向信息检索的对抗性模仿点击模型

An Adversarial Imitation Click Model for Information Retrieval

Personalization is a crucial aspect of many online experiences. In
particular, content ranking is often a key component in delivering
sophisticated personalization results. Commonly, supervised learning-to-rank
methods are applied, which suffer from bias introduced during data collection
by production systems in charge of producing the ranking. To compensate for
this problem, we leverage contextual multi-armed bandits. We propose novel
extensions of two well-known algorithms viz. LinUCB and Linear Thompson
Sampling to the ranking use-case. To account for the biases in a production
environment, we employ the position-based click model. Finally, we show the
validity of the proposed algorithms by conducting extensive offline experiments
on synthetic datasets as well as customer facing online A/B experiments.

本文介绍了一种基于多臂老虎机算法和位置点击模型的方法，用于解决线上系统中推荐排名的偏见问题，并在合成和实际数据集的离线和在线 A/B 测试中证明了该算法的有效性。