Tactics, Techniques, and Procedures (TTPs) outline the methods attackers use
to exploit vulnerabilities. The interpretation of TTPs in the MITRE ATT&CK
framework can be challenging for cybersecurity practitioners due to presumed
expertise, complex dependencies, and inherent ambiguity. Meanwhile,
advancements with Large Language Models (LLMs) have led to recent surge in
studies exploring its uses in cybersecurity operations. This leads us to
question how well encoder-only (e.g., RoBERTa) and decoder-only (e.g., GPT-3.5)
LLMs can comprehend and summarize TTPs to inform analysts of the intended
purposes (i.e., tactics) of a cyberattack procedure. The state-of-the-art LLMs
have shown to be prone to hallucination by providing inaccurate information,
which is problematic in critical domains like cybersecurity. Therefore, we
propose the use of Retrieval Augmented Generation (RAG) techniques to extract
relevant contexts for each cyberattack procedure for decoder-only LLMs (without
fine-tuning). We further contrast such approach against supervised fine-tuning
(SFT) of encoder-only LLMs. Our results reveal that both the direct-use of
decoder-only LLMs (i.e., its pre-trained knowledge) and the SFT of encoder-only
LLMs offer inaccurate interpretation of cyberattack procedures. Significant
improvements are shown when RAG is used for decoder-only LLMs, particularly
when directly relevant context is found. This study further sheds insights on
the limitations and capabilities of using RAG for LLMs in interpreting TTPs.

使用编码器 - 解码器结构的大型语言模型（LLM）在解释攻击者行动方法（Tactics, Techniques, and Procedures）时存在问题，为此我们提出使用检索增强生成（Retrieval Augmented Generation）技术来提取相关上下文以改进解码器 - LLM 的表现。研究结果显示，直接使用解码器 - LLM 和仅对编码器 - LLM 进行监督微调的方法都无法准确解释网络攻击步骤，而将检索增强生成技术应用于解码器 - LLM 则显著提升了解释能力。本研究进一步揭示了在解释行动方法时使用检索增强生成技术的限制和能力。