In this paper, we present a new reinforcement learning (RL) algorithm called
Distributional Soft Actor Critic (DSAC), which exploits the distributional
information of accumulated rewards to achieve better performance. Seamlessly
integrating SAC (which uses entropy to encourage exploration) with a principled
distributional view of the underlying objective, DSAC takes into consideration
the randomness in both action and rewards, and beats the state-of-the-art
baselines in several continuous control benchmarks. Moreover, with the
distributional information of rewards, we propose a unified framework for
risk-sensitive learning, one that goes beyond maximizing only expected
accumulated rewards. Under this framework we discuss three specific
risk-related metrics: percentile, mean-variance and distorted expectation. Our
extensive experiments demonstrate that with distribution modeling in RL, the
agent performs better for both risk-averse and risk-seeking control tasks.

DSAC 是一种新的强化学习算法，它通过利用积累奖励的分布信息来获得更好的性能。通过将 SAC 与基本分布式目标观点无缝集成， DSAC 考虑了行动和回报中的随机性，并在几个连续控制基准测试中超越了现有技术基线。除此之外，我们还探讨了三个具体的与风险相关的度量标准：百分位数，均值 - 方差和扭曲期望，通过分布建模实现了 RL 中的风险敏感。