Most reinforcement learning algorithms are based on a key assumption that
Markov decision processes (MDPs) are stationary. However, non-stationary MDPs
with dynamic action space are omnipresent in real-world scenarios. Yet problems
of dynamic action space reinforcement learning have been studied by many
previous works, how to choose valuable actions from new and unseen actions to
improve learning efficiency remains unaddressed. To tackle this problem, we
propose an intelligent Action Pick-up (AP) algorithm to autonomously choose
valuable actions that are most likely to boost performance from a set of new
actions. In this paper, we first theoretically analyze and find that a prior
optimal policy plays an important role in action pick-up by providing useful
knowledge and experience. Then, we design two different AP methods:
frequency-based global method and state clustering-based local method, based on
the prior optimal policy. Finally, we evaluate the AP on two simulated but
challenging environments where action spaces vary over time. Experimental
results demonstrate that our proposed AP has advantages over baselines in
learning efficiency.

提出了一种智能的 Action Pick-up 算法，用于从新的和未知的动作中自主选择最有可能提高性能的有价值的动作，它利用之前的最优策略提供有用的知识和经验，具有比基线更优的学习效率。