This work pioneers regret analysis of risk-sensitive reinforcement learning
in partially observable environments with hindsight observation, addressing a
gap in theoretical exploration. We introduce a novel formulation that
integrates hindsight observations into a Partially Observable Markov Decision
Process (POMDP) framework, where the goal is to optimize accumulated reward
under the entropic risk measure. We develop the first provably efficient RL
algorithm tailored for this setting. We also prove by rigorous analysis that
our algorithm achieves polynomial regret
$\tilde{O}\left(\frac{e^{|{\gamma}|H}-1}{|{\gamma}|H}H^2\sqrt{KHS^2OA}\right)$,
which outperforms or matches existing upper bounds when the model degenerates
to risk-neutral or fully observable settings. We adopt the method of
change-of-measure and develop a novel analytical tool of beta vectors to
streamline mathematical derivations. These techniques are of particular
interest to the theoretical study of reinforcement learning.

该论文通过引入后见观察机制，研究了部分可观测环境下风险敏感强化学习的悔恨分析，提出了在部分可观测马尔可夫决策过程框架下优化累积奖励的新方法。通过严格的分析证明了算法在模型降级为风险中性或完全可观测设置时，能够实现多项式悔恨。该研究对强化学习的理论研究具有特殊意义。