Temporal credit assignment is crucial for learning and skill development in
natural and artificial intelligence. While computational methods like the TD
approach in reinforcement learning have been proposed, it's unclear if they
accurately represent how humans handle feedback delays. Cognitive models intend
to represent the mental steps by which humans solve problems and perform a
number of tasks, but limited research in cognitive science has addressed the
credit assignment problem in humans and cognitive models. Our research uses a
cognitive model based on a theory of decisions from experience, Instance-Based
Learning Theory (IBLT), to test different credit assignment mechanisms in a
goal-seeking navigation task with varying levels of decision complexity.
Instance-Based Learning (IBL) models simulate the process of making sequential
choices with different credit assignment mechanisms, including a new IBL-TD
model that combines the IBL decision mechanism with the TD approach. We found
that (1) An IBL model that gives equal credit assignment to all decisions is
able to match human performance better than other models, including IBL-TD and
Q-learning; (2) IBL-TD and Q-learning models underperform compared to humans
initially, but eventually, they outperform humans; (3) humans are influenced by
decision complexity, while models are not. Our study provides insights into the
challenges of capturing human behavior and the potential opportunities to use
these models in future AI systems to support human activities.

本研究使用基于经验决策的认知模型 IBLT 来测试不同的信用分配机制，并发现决策复杂性会影响人类表现，而不影响模型的表现。机器学习方法如 TD 方法和 Q 学习方法在人类能力匹配方面表现不佳，但在最终能够超越人类。IBL 模型在信用分配方面比其他模型表现更好，研究为未来 AI 系统的开发提供了洞察。

信用分配：发展类人智能代理人的挑战和机遇

Credit Assignment: Challenges and Opportunities in Developing Human-like  AI Agents

The increasing use of machine learning in high-stakes domains -- where
people's livelihoods are impacted -- creates an urgent need for interpretable,
fair, and highly accurate algorithms. With these needs in mind, we propose a
mixed integer optimization (MIO) framework for learning optimal classification
trees -- one of the most interpretable models -- that can be augmented with
arbitrary fairness constraints. In order to better quantify the "price of
interpretability", we also propose a new measure of model interpretability
called decision complexity that allows for comparisons across different classes
of machine learning models. We benchmark our method against state-of-the-art
approaches for fair classification on popular datasets; in doing so, we conduct
one of the first comprehensive analyses of the trade-offs between
interpretability, fairness, and predictive accuracy. Given a fixed disparity
threshold, our method has a price of interpretability of about 4.2 percentage
points in terms of out-of-sample accuracy compared to the best performing,
complex models. However, our method consistently finds decisions with almost
full parity, while other methods rarely do.

本研究设计了一个 MIO（mixed integer optimization）框架，用于学习最优的分类树算法，以及与任意公平性约束进行扩展；提出了一个新的模型可解释性度量标准，称为决策复杂度；在流行数据集上，我们对公平性、可解释性及预测准确性之间的权衡进行了综合分析，证明了我们的方法在几乎完全平等的情况下仍能保持精度.