Modern deep learning models often make predictions by focusing on irrelevant areas, leading to biased performance and limited generalization. Existing methods aimed at rectifying model attention require explicit labels for irrelevant areas or complex pixel-wise ground truth attention maps. We present CRAYON (Correcting Reasoning with Annotations of Yes Or No), offering effective, scalable, and practical solutions to rectify model attention using simple yes-no annotations. CRAYON empowers classical and modern model interpretation techniques to identify and guide model reasoning: CRAYON-ATTENTION directs classic interpretations based on saliency maps to focus on relevant image regions, while CRAYON-PRUNING removes irrelevant neurons identified by modern concept-based methods to mitigate their influence. Through extensive experiments with both quantitative and human evaluation, we showcase CRAYON's effectiveness, scalability, and practicality in refining model attention. CRAYON achieves state-of-the-art performance, outperforming 12 methods across 3 benchmark datasets, surpassing approaches that require more complex annotations.

本研究解决了深度学习模型在预测时关注无关区域的问题，导致性能偏差和泛化能力有限。通过提出CRAYON（用是或否注释纠正推理），我们提供了有效且可扩展的解决方案，利用简单的是非注释来修正模型注意力。实验结果表明，CRAYON在精炼模型注意力方面的效果显著，超过了12种方法，并在3个基准数据集上达到了最先进的性能。

有效指导模型注意力的简单是非注释