This paper explores continuous-time control synthesis for target-driven
navigation to satisfy complex high-level tasks expressed as linear temporal
logic (LTL). We propose a model-free framework using deep reinforcement
learning (DRL) where the underlying dynamic system is unknown (an opaque box).
Unlike prior work, this paper considers scenarios where the given LTL
specification might be infeasible and therefore cannot be accomplished
globally. Instead of modifying the given LTL formula, we provide a general
DRL-based approach to satisfy it with minimal violation. To do this, we
transform a previously multi-objective DRL problem, which requires simultaneous
automata satisfaction and minimum violation cost, into a single objective. By
guiding the DRL agent with a sampling-based path planning algorithm for the
potentially infeasible LTL task, the proposed approach mitigates the myopic
tendencies of DRL, which are often an issue when learning general LTL tasks
that can have long or infinite horizons. This is achieved by decomposing an
infeasible LTL formula into several reach-avoid sub-tasks with shorter
horizons, which can be trained in a modular DRL architecture. Furthermore, we
overcome the challenge of the exploration process for DRL in complex and
cluttered environments by using path planners to design rewards that are dense
in the configuration space. The benefits of the presented approach are
demonstrated through testing on various complex nonlinear systems and compared
with state-of-the-art baselines. The Video demonstration can be found
here:this https URL.

本文研究了基于深度强化学习的模型无关框架来解决复杂高级任务下的连续时间控制综合问题，并提出了一种基于采样路径规划算法来满足要求的方案，旨在降低深度强化学习在解决长期任务时的局限性和探索复杂环境问题