In many problems, it is desirable to optimize an objective function while
imposing constraints on some other aspect of the problem. A Constrained
Partially Observable Markov Decision Process (C-POMDP) allows modelling of such
problems while subject to transition uncertainty and partial observability.
Typically, the constraints in C-POMDPs enforce a threshold on expected
cumulative costs starting from an initial state distribution. In this work, we
first show that optimal C-POMDP policies may violate Bellman's principle of
optimality and thus may exhibit pathological behaviors, which can be
undesirable for many applications. To address this drawback, we introduce a new
formulation, the Recursively-Constrained POMDP (RC-POMDP), that imposes
additional history dependent cost constraints on the C-POMDP. We show that,
unlike C-POMDPs, RC-POMDPs always have deterministic optimal policies, and that
optimal policies obey Bellman's principle of optimality. We also present a
point-based dynamic programming algorithm that synthesizes optimal policies for
RC-POMDPs. In our evaluations, we show that policies for RC-POMDPs produce more
desirable behavior than policies for C-POMDPs and demonstrate the efficacy of
our algorithm across a set of benchmark problems.

通过引入新的限制性、历史依赖成本约束的递归约束部分可观察马尔可夫决策问题 (RC-POMDP)，本文解决了常规约束部分可观察马尔可夫决策问题 (C-POMDP) 中存在的问题，并提出了一个基于点的动态规划算法来寻找 RC-POMDP 的最优策略。实验证明，相比于 C-POMDP 的策略，RC-POMDP 的策略具有更好的行为，并展示了算法在一组基准问题上的有效性。