We consider the problem of creating assistants that can help agents - often humans - solve novel sequential decision problems, assuming the agent is not able to specify the reward function explicitly to the assistant. Instead of aiming to automate, and act in place of the agent as in current approaches, we give the assistant an advisory role and keep the agent in the loop as the main decision maker. The difficulty is that we must account for potential biases induced by limitations or constraints of the agent which may cause it to seemingly irrationally reject advice. To do this we introduce a novel formalization of assistance that models these biases, allowing the assistant to infer and adapt to them. We then introduce a new method for planning the assistant's advice which can scale to large decision making problems. Finally, we show experimentally that our approach adapts to these agent biases, and results in higher cumulative reward for the agent than automation-based alternatives.

本文研究如何创建助手来帮助代理人解决新的顺序决策问题，介绍了一种新的辅助形式来模拟代理人的偏见，并提出了一种新的规划方法来缩放大型的决策问题，目的是为了获得比基于自动化的替代方案更高的累计奖励，最后我们证明了将建议和自动化相结合的方法比仅使用建议具有更好的性能，尽管会失去一些安全保证。

序列决策问题中的零-shot辅助