Some worry that advanced artificial agents may resist being shut down. The Incomplete Preferences Proposal (IPP) is an idea for ensuring that doesn't happen. A key part of the IPP is using a novel 'Discounted REward for Same-Length Trajectories (DREST)' reward function to train agents to (1) pursue goals effectively conditional on each trajectory-length (be 'USEFUL'), and (2) choose stochastically between different trajectory-lengths (be 'NEUTRAL' about trajectory-lengths). In this paper, we propose evaluation metrics for USEFULNESS and NEUTRALITY. We use a DREST reward function to train simple agents to navigate gridworlds, and we find that these agents learn to be USEFUL and NEUTRAL. Our results thus suggest that DREST reward functions could also train advanced agents to be USEFUL and NEUTRAL, and thereby make these advanced agents useful and shutdownable.

提出了一种使用 Discounted REward for Same-Length Trajectories (DREST) 奖励函数的不完全偏好方案，以训练人工智能代理，使其既追求目标而且中立对待轨迹长度。实验结果表明，DREST 奖励函数能够使简单代理在网络中变得有用且对轨迹长度保持中立，进一步证明该函数可以用于训练先进的代理，使其成为有用且可关闭的。

通过随机选择实现可关闭的智能体