We explored leveraging state-of-the-art deep learning, big data, and natural
language processing to enhance the detection of vulnerable web server versions.
Focusing on improving accuracy and specificity over rule-based systems, we
conducted experiments by sending various ambiguous and non-standard HTTP
requests to 4.77 million domains and capturing HTTP response status lines. We
represented these status lines through training a BPE tokenizer and RoBERTa
encoder for unsupervised masked language modeling. We then dimensionality
reduced and concatenated encoded response lines to represent each domain's web
server. A Random Forest and multilayer perceptron (MLP) classified these web
servers, and achieved 0.94 and 0.96 macro F1-score, respectively, on detecting
the five most popular origin web servers. The MLP achieved a weighted F1-score
of 0.55 on classifying 347 major type and minor version pairs. Analysis
indicates that our test cases are meaningful discriminants of web server types.
Our approach demonstrates promise as a powerful and flexible alternative to
rule-based systems.

利用最先进的深度学习、大数据和自然语言处理技术探索提高对易受攻击的 Web 服务器版本检测的方法，通过发送各种模糊和非标准的 HTTP 请求到 477 万个域名并捕获 HTTP 响应状态行，通过使用 BPE 标记器和 RoBERTa 编码器进行无监督的掩码语言建模来表示这些状态行，对编码的响应行进行降维和连接以表示每个域的 Web 服务器，通过 Random Forest 和多层感知器（MLP）对这些 Web 服务器进行分类，检测到五种最流行的起点 Web 服务器的宏 F1 分数分别为 0.94 和 0.96，MLP 在对 347 种主要类型和次要版本对进行分类时得到了权重 F1 分数 0.55，分析表明我们的测试案例是 Web 服务器类型的有意义的差异判别因子，我们的方法展示了作为基于规则系统的强大而灵活的替代方案的潜力。