In deep reinforcement learning applications, maximizing discounted reward is often employed instead of maximizing total reward to ensure the convergence and stability of algorithms, even though the performance metric for evaluating the policy remains the total reward. However, the optimal policies corresponding to these two objectives may not always be consistent. To address this issue, we analyzed the suboptimality of the policy obtained through maximizing discounted reward in relation to the policy that maximizes total reward and identified the influence of hyperparameters. Additionally, we proposed sufficient conditions for aligning the optimal policies of these two objectives under various settings. The primary contributions are as follows: We theoretically analyzed the factors influencing performance when using discounted reward as a proxy for total reward, thereby enhancing the theoretical understanding of this scenario. Furthermore, we developed methods to align the optimal policies of the two objectives in certain situations, which can improve the performance of reinforcement learning algorithms.

在深度强化学习应用中，通过最大化折扣奖励而不是最大化总奖励来确保算法的收敛性和稳定性，尽管评估策略的性能指标仍然是总奖励。然而，这两个目标对应的最优策略可能并不总是一致的。为了解决这个问题，我们分析了通过最大化折扣奖励得到的策略与最大化总奖励得到的策略之间的次优性，并确定了超参数的影响。此外，我们提出了在各种情况下使这两个目标的最优策略保持一致的充分条件。主要贡献如下：我们在理论上分析了当使用折扣奖励作为总奖励的代理时影响性能的因素，从而增强了对这种情况的理论理解。此外，我们开发了在某些情况下使这两个目标的最优策略保持一致的方法，可以提高强化学习算法的性能。

深度强化学习中最大化总奖励与折扣奖励之间的分析与弥合