Many practical applications, such as recommender systems and learning to
rank, involve solving multiple similar tasks. One example is learning of
recommendation policies for users with similar movie preferences, where the
users may still rank the individual movies slightly differently. Such tasks can
be organized in a hierarchy, where similar tasks are related through a shared
structure. In this work, we formulate this problem as a contextual off-policy
optimization in a hierarchical graphical model from logged bandit feedback. To
solve the problem, we propose a hierarchical off-policy optimization algorithm
(HierOPO), which estimates the parameters of the hierarchical model and then
acts pessimistically with respect to them. We instantiate HierOPO in linear
Gaussian models, for which we also provide an efficient implementation and
analysis. We prove per-task bounds on the suboptimality of the learned
policies, which show a clear improvement over not using the hierarchical model.
We also evaluate the policies empirically. Our theoretical and empirical
results show a clear advantage of using the hierarchy over solving each task
independently.

提出针对多个类似问题的层次图模型，用基于带环反馈的上下文离线策略优化解决问题，提出 HierOPO 算法并通过线性高斯模型实现，在模型、理论和实证数据上比单独解决每个任务有优势。