Partially observable Markov decision processes (POMDPs) provide a flexible
representation for real-world decision and control problems. However, POMDPs
are notoriously difficult to solve, especially when the state and observation
spaces are continuous or hybrid, which is often the case for physical systems.
While recent online sampling-based POMDP algorithms that plan with observation
likelihood weighting have shown practical effectiveness, a general theory
characterizing the approximation error of the particle filtering techniques
that these algorithms use has not previously been proposed. Our main
contribution is bounding the error between any POMDP and its corresponding
finite sample particle belief MDP (PB-MDP) approximation. This fundamental
bridge between PB-MDPs and POMDPs allows us to adapt any sampling-based MDP
algorithm to a POMDP by solving the corresponding particle belief MDP, thereby
extending the convergence guarantees of the MDP algorithm to the POMDP.
Practically, this is implemented by using the particle filter belief transition
model as the generative model for the MDP solver. While this requires access to
the observation density model from the POMDP, it only increases the transition
sampling complexity of the MDP solver by a factor of $\mathcal{O}(C)$, where
$C$ is the number of particles. Thus, when combined with sparse sampling MDP
algorithms, this approach can yield algorithms for POMDPs that have no direct
theoretical dependence on the size of the state and observation spaces. In
addition to our theoretical contribution, we perform five numerical experiments
on benchmark POMDPs to demonstrate that a simple MDP algorithm adapted using
PB-MDP approximation, Sparse-PFT, achieves performance competitive with other
leading continuous observation POMDP solvers.

本文提出了利用基于粒子滤波置信转移模型的有限样本粒子置信 MDP 近似解决 POMDP 的方法。在五个基准 POMDP 实验中，与其他领先的连续观察 POMDP 求解器相比，表明这种方法可以实现与其他领先的连续观察 POMDP 求解器竞争力的表现。