Partially Observable Markov Decision Processes (POMDPs) are powerful models
for sequential decision making under transition and observation uncertainties.
This paper studies the challenging yet important problem in POMDPs known as the
(indefinite-horizon) Maximal Reachability Probability Problem (MRPP), where the
goal is to maximize the probability of reaching some target states. This is
also a core problem in model checking with logical specifications and is
naturally undiscounted (discount factor is one). Inspired by the success of
point-based methods developed for discounted problems, we study their
extensions to MRPP. Specifically, we focus on trial-based heuristic search
value iteration techniques and present a novel algorithm that leverages the
strengths of these techniques for efficient exploration of the belief space
(informed search via value bounds) while addressing their drawbacks in handling
loops for indefinite-horizon problems. The algorithm produces policies with
two-sided bounds on optimal reachability probabilities. We prove convergence to
an optimal policy from below under certain conditions. Experimental evaluations
on a suite of benchmarks show that our algorithm outperforms existing methods
in almost all cases in both probability guarantees and computation time.

该研究论文探讨了部分可观测马尔可夫决策过程（POMDPs）中的最大可达性概率问题（MRPP），研究了一种基于点估计方法和基于试探性启发式搜索的新算法，解决了处理无限时域问题中循环的问题，并在实验评估中表现出优越性。

无折扣 POMDP 中带有可达性目标的声音启发搜索值迭代

Sound Heuristic Search Value Iteration for Undiscounted POMDPs with  Reachability Objectives

Partially-observable problems pose a trade-off between reducing costs and
gathering information. They can be solved optimally by planning in belief
space, but that is often prohibitively expensive. Model-predictive control
(MPC) takes the alternative approach of using a state estimator to form a
belief over the state, and then plan in state space. This ignores potential
future observations during planning and, as a result, cannot actively increase
or preserve the certainty of its own state estimate. We find a middle-ground
between planning in belief space and completely ignoring its dynamics by only
reasoning about its future accuracy. Our approach, filter-aware MPC, penalises
the loss of information by what we call "trackability", the expected error of
the state estimator. We show that model-based simulation allows condensing
trackability into a neural network, which allows fast planning. In experiments
involving visual navigation, realistic every-day environments and a two-link
robot arm, we show that filter-aware MPC vastly improves regular MPC.

本文采用滤波器感知的模型预测控制方法，既考虑了不确定性的影响，又可以通过建立一个包含感知能力的模型来显著提高控制性能。

滤波感知模型预测控制

Filter-Aware Model-Predictive Control

Planning robust executions under uncertainty is a fundamental challenge for
building autonomous robots. Partially Observable Markov Decision Processes
(POMDPs) provide a standard framework for modeling uncertainty in many
applications. In this work, we study POMDPs with safe-reachability objectives,
which require that with a probability above some threshold, a goal state is
eventually reached while keeping the probability of visiting unsafe states
below some threshold. This POMDP formulation is different from the traditional
POMDP models with optimality objectives and we show that in some cases, POMDPs
with safe-reachability objectives can provide a better guarantee of both safety
and reachability than the existing POMDP models through an example. A key
algorithmic problem for POMDPs is policy synthesis, which requires reasoning
over a vast space of beliefs (probability distributions). To address this
challenge, we introduce the notion of a goal-constrained belief space, which
only contains beliefs reachable from the initial belief under desired
executions that can achieve the given safe-reachability objective. Our method
compactly represents this space over a bounded horizon using symbolic
constraints, and employs an incremental Satisfiability Modulo Theories (SMT)
solver to efficiently search for a valid policy over it. We evaluate our method
using a case study involving a partially observable robotic domain with
uncertain obstacles. The results show that our method can synthesize policies
over large belief spaces with a small number of SMT solver calls by focusing on
the goal-constrained belief space.

本文研究了带安全可达性目标的部分可观测马尔可夫决策过程（POMDPs），提出了一种基于目标约束信念空间和符号约束的方法来合成能实现安全可达性目标的策略，并通过实验结果表明，该方法能够在大量信念空间中高效地搜索有效策略。