Binary code is pervasive, and binary analysis is a key task in reverse
engineering, malware classification, and vulnerability discovery.
Unfortunately, while there exist large corpuses of malicious binaries,
obtaining high-quality corpuses of benign binaries for modern systems has
proven challenging (e.g., due to licensing issues). Consequently, machine
learning based pipelines for binary analysis utilize either costly commercial
corpuses (e.g., VirusTotal) or open-source binaries (e.g., coreutils) available
in limited quantities. To address these issues, we present Assemblage: an
extensible cloud-based distributed system that crawls, configures, and builds
Windows PE binaries to obtain high-quality binary corpuses suitable for
training state-of-the-art models in binary analysis. We have run Assemblage on
AWS over the past year, producing 890k Windows PE and 428k Linux ELF binaries
across 29 configurations. Assemblage is designed to be both reproducible and
extensible, enabling users to publish "recipes" for their datasets, and
facilitating the extraction of a wide array of features. We evaluated
Assemblage by using its data to train modern learning-based pipelines for
compiler provenance and binary function similarity. Our results illustrate the
practical need for robust corpuses of high-quality Windows PE binaries in
training modern learning-based binary analyses. Assemblage can be downloaded
from this https URL

Assemblage 是一个可扩展的基于云的分布式系统，用于获取高质量的二进制语料库，以训练二进制分析的先进模型。它可以通过爬取、配置和构建 Windows PE 二进制文件来获取高质量的二进制语料库，并支持用户发布数据集的 “recipes” 和提取各种特征。