Action-value estimation is a critical component of many reinforcement
learning (RL) methods whereby sample complexity relies heavily on how fast a
good estimator for action value can be learned. By viewing this problem through
the lens of representation learning, good representations of both state and
action can facilitate action-value estimation. While advances in deep learning
have seamlessly driven progress in learning state representations, given the
specificity of the notion of agency to RL, little attention has been paid to
learning action representations. We conjecture that leveraging the
combinatorial structure of multi-dimensional action spaces is a key ingredient
for learning good representations of action. To test this, we set forth the
action hypergraph networks framework -- a class of functions for learning
action representations in multi-dimensional discrete action spaces with a
structural inductive bias. Using this framework we realise an agent class based
on a combination with deep Q-networks, which we dub hypergraph Q-networks. We
show the effectiveness of our approach on a myriad of domains: illustrative
prediction problems under minimal confounding effects, Atari 2600 games, and
discretised physical control benchmarks.

通过结合 hypergraph networks framework 和 deep Q-networks 方法，有效提升异构空间下 action-value 估计的表现，包括 Atari 2600 游戏和物理控制基准测试等多个领域。