We consider the problem of learning a set of probability distributions from
the empirical Bellman dynamics in distributional reinforcement learning (RL), a
class of state-of-the-art methods that estimate the distribution, as opposed to
only the expectation, of the total return. We form