The recommender system is an important form of intelligent application, which
assists users to alleviate from information redundancy. Among the metrics used
to evaluate a recommender system, the metric of conversion has become more and
more important. The majority of existing recommender systems perform poorly on
the metric of conversion due to its extremely sparse feedback signal. To tackle
this challenge, we propose a deep hierarchical reinforcement learning based
recommendation framework, which consists of two components, i.e., high-level
agent and low-level agent. The high-level agent catches long-term sparse
conversion signals, and automatically sets abstract goals for low-level agent,
while the low-level agent follows the abstract goals and interacts with
real-time environment. To solve the inherent problem in hierarchical
reinforcement learning, we propose a novel deep hierarchical reinforcement
learning algorithm via multi-goals abstraction (HRL-MG). Our proposed algorithm
contains three characteristics: 1) the high-level agent generates multiple
goals to guide the low-level agent in different stages, which reduces the
difficulty of approaching high-level goals; 2) different goals share the same
state encoder parameters, which increases the update frequency of the
high-level agent and thus accelerates the convergence of our proposed
algorithm; 3) an appreciate benefit assignment function is designed to allocate
rewards in each goal so as to coordinate different goals in a consistent
direction. We evaluate our proposed algorithm based on a real-world e-commerce
dataset and validate its effectiveness.

该论文提出一种基于分层强化学习的推荐框架，其中高层代理获取长期稀疏的转化信号并设置抽象目标，而低层代理根据抽象目标与实时环境交互，提出了多目标抽象的深层分层强化学习算法并验证其有效性。

基于深层次层次强化学习的多目标抽象推荐

Deep Hierarchical Reinforcement Learning Based Recommendations via  Multi-goals Abstraction

Recently, neural models for information retrieval are becoming increasingly
popular. They provide effective approaches for product search due to their
competitive advantages in semantic matching. However, it is challenging to use
graph-based features, though proved very useful in IR literature, in these
neural approaches. In this paper, we leverage the recent advances in graph
embedding techniques to enable neural retrieval models to exploit
graph-structured data for automatic feature extraction. The proposed approach
can not only help to overcome the long-tail problem of click-through data, but
also incorporate external heterogeneous information to improve search results.
Extensive experiments on a real-world e-commerce dataset demonstrate
significant improvement achieved by our proposed approach over multiple strong
baselines both as an individual retrieval model and as a feature used in
learning-to-rank frameworks.

本文介绍了一种利用图嵌入技术使神经信息检索模型可以利用图结构数据进行自动特征提取的方法，并将其应用于电子商务数据集中，结果显示该方法在多个强基线模型和排序框架中都有显著提高。

神经信息检索与图嵌入相遇：产品搜索排序模型

Neural IR Meets Graph Embedding: A Ranking Model for Product Search

Recommender systems can mitigate the information overload problem by
suggesting users' personalized items. In real-world recommendations such as
e-commerce, a typical interaction between the system and its users is -- users
are recommended a page of items and provide feedback; and then the system
recommends a new page of items. To effectively capture such interaction for
recommendations, we need to solve two key problems -- (1) how to update
recommending strategy according to user's \textit{real-time feedback}, and 2)
how to generate a page of items with proper display, which pose tremendous
challenges to traditional recommender systems. In this paper, we study the
problem of page-wise recommendations aiming to address aforementioned two
challenges simultaneously. In particular, we propose a principled approach to
jointly generate a set of complementary items and the corresponding strategy to
display them in a 2-D page; and propose a novel page-wise recommendation
framework based on deep reinforcement learning, DeepPage, which can optimize a
page of items with proper display based on real-time feedback from users. The
experimental results based on a real-world e-commerce dataset demonstrate the
effectiveness of the proposed framework.

本文提出了一个基于深度强化学习的新型页面推荐框架 ——DeepPage，旨在同时解决如何根据用户实时反馈更新推荐策略以及如何生成具有适当显示的项目页面两个关键问题，并在真实世界的电子商务数据集上进行实验证明了提出框架的有效性。