Binary Code Embedding (BCE) has important applications in various reverse engineering tasks such as binary code similarity detection, type recovery, control-flow recovery and data-flow analysis. Recent studies have shown that the Transformer model can comprehend the semantics of binary code to support downstream tasks. However, existing models overlooked the prior knowledge of assembly language. In this paper, we propose a novel Transformer-based approach, namely kTrans, to generate knowledge-aware binary code embedding. By feeding explicit knowledge as additional inputs to the Transformer, and fusing implicit knowledge with a novel pre-training task, kTrans provides a new perspective to incorporating domain knowledge into a Transformer framework. We inspect the generated embeddings with outlier detection and visualization, and also apply kTrans to 3 downstream tasks: Binary Code Similarity Detection (BCSD), Function Type Recovery (FTR) and Indirect Call Recognition (ICR). Evaluation results show that kTrans can generate high-quality binary code embeddings, and outperforms state-of-the-art (SOTA) approaches on downstream tasks by 5.2%, 6.8%, and 12.6% respectively. kTrans is publicly available at: https://github.com/Learner0x5a/kTrans-release

我们提出了一种基于Transformer的方法，即kTrans，用于生成具有知识感知能力的二进制代码嵌入。通过将显式知识作为额外输入输入到Transformer中，并通过一种新的预训练任务将隐式知识与之融合，kTrans为将领域知识融入Transformer框架提供了新的视角。评估结果显示，kTrans可以生成高质量的二进制代码嵌入，且在下游任务中表现优于最先进方法分别达到5.2％，6.8％和12.6％。

kTrans: 知识感知Transformer用于二进制代码嵌入