We introduce phasic policy gradient (PPG), a reinforcement learning framework which modifies traditional on-policy actor-critic methods by