Reusing previously trained models is critical in deep reinforcement learning
to speed up training of new agents. However, it is unclear how to acquire new
skills when objectives and constraints are in conflict with previously learned
skills. Moreover, when retraining, there is an intrinsic conflict between
exploiting what has already been learned and exploring new skills. In soft
actor-critic (SAC) methods, a temperature parameter can be dynamically adjusted
to weight the action entropy and balance the explore $\times$ exploit
trade-off. However, controlling a single coefficient can be challenging within
the context of retraining, even more so when goals are contradictory. In this
work, inspired by neuroscience research, we propose a novel approach using
inhibitory networks to allow separate and adaptive state value evaluations, as
well as distinct automatic entropy tuning. Ultimately, our approach allows for
controlling inhibition to handle conflict between exploiting less risky,
acquired behaviors and exploring novel ones to overcome more challenging tasks.
We validate our method through experiments in OpenAI Gym environments.

本文提出了一种受神经科学研究启发的新方法，使用抑制性网络，以允许分离和自适应状态值评估，以及不同的自动熵调整，最终我们的方法允许控制抑制来处理利用较少风险获取的行为和探索新行为之间的冲突，以克服更具挑战性的任务，并通过在 OpenAI Gym 环境中进行实验验证。