Domain-specific language understanding requires integrating multiple pieces
of relevant contextual information. For example, we see both suicide and
depression-related behavior (multiple contexts) in the text ``I have a gun and
feel pretty bad about my life, and it wouldn't be the worst thing if I didn't
wake up tomorrow''. Domain specificity in self-attention architectures is
handled by fine-tuning on excerpts from relevant domain specific resources
(datasets and external knowledge - medical textbook chapters on mental health
diagnosis related to suicide and depression). We propose a modified
self-attention architecture Knowledge-infused Self Attention Transformer (KSAT)
that achieves the integration of multiple domain-specific contexts through the
use of external knowledge sources. KSAT introduces knowledge-guided biases in
dedicated self-attention layers for each knowledge source to accomplish this.
In addition, KSAT provides mechanics for controlling the trade-off between
learning from data and learning from knowledge. Our quantitative and
qualitative evaluations show that (1) the KSAT architecture provides novel
human-understandable ways to precisely measure and visualize the contributions
of the infused domain contexts, and (2) KSAT performs competitively with other
knowledge-infused baselines and significantly outperforms baselines that use
fine-tuning for domain-specific tasks.

通过使用来自相关领域的外部知识资源，在 KSAT 中引入了知识注入的自我关注层，从而实现了对多个领域特定上下文的集成。KSAT 提供了控制从数据中学习与从知识中学习之间的权衡的机制，并与其他知识注入基线相竞争，显着优于使用精调进行领域特定任务的基线。