In this paper, we study Contextual Unsupervised Sequential Selection (USS), a
new variant of the stochastic contextual bandits problem where the loss of an
arm cannot be inferred from the observed feedback. In our setup, arms are
associated with fixed costs and are ordered, forming a cascade. In each round,
a context is presented, and the learner selects the arms sequentially till some
depth. The total cost incurred by stopping at an arm is the sum of fixed costs
of arms selected and the stochastic loss associated with the arm. The learner's
goal is to learn a decision rule that maps contexts to arms with the goal of
minimizing the total expected loss. The problem is challenging as we are faced
with an unsupervised setting as the total loss cannot be estimated. Clearly,
learning is feasible only if the optimal arm can be inferred (explicitly or
implicitly) from the problem structure. We observe that learning is still
possible when the problem instance satisfies the so-called 'Contextual Weak
Dominance' (CWD) property. Under CWD, we propose an algorithm for the
contextual USS problem and demonstrate that it has sub-linear regret.
Experiments on synthetic and real datasets validate our algorithm.

研究了一个新的随机上下文臂问题变体，其中臂的损失无法从观察到的反馈中推断出来，每一轮中介绍一个上下文，学习者将臂顺序地选择到一定深度，如果问题结构中的最优臂可以得到明确或隐含的推断，则可以通过提出的算法来进行学习并展示它具有次线性的遗憾。