Malicious URLs host unsolicited content and are used to perpetrate
cybercrimes. It is imperative to detect them in a timely manner. Traditionally,
this is done through the usage of blacklists, which cannot be exhaustive, and
cannot detect newly generated malicious URLs. To address this, recent years
have witnessed several efforts to perform Malicious URL Detection using Machine
Learning. The most popular and scalable approaches use lexical properties of
the URL string by extracting Bag-of-words like features, followed by applying
machine learning models such as SVMs. There are also other features designed by
experts to improve the prediction performance of the model. These approaches
suffer from several limitations: (i) Inability to effectively capture semantic
meaning and sequential patterns in URL strings; (ii) Requiring substantial
manual feature engineering; and (iii) Inability to handle unseen features and
generalize to test data. To address these challenges, we propose URLNet, an
end-to-end deep learning framework to learn a nonlinear URL embedding for
Malicious URL Detection directly from the URL. Specifically, we apply
Convolutional Neural Networks to both characters and words of the URL String to
learn the URL embedding in a jointly optimized framework. This approach allows
the model to capture several types of semantic information, which was not
possible by the existing models. We also propose advanced word-embeddings to
solve the problem of too many rare words observed in this task. We conduct
extensive experiments on a large-scale dataset and show a significant
performance gain over existing methods. We also conduct ablation studies to
evaluate the performance of various components of URLNet.

本文介绍 URLNet，一种基于端到端深度学习的框架，可直接从 URL 中学习非线性嵌入以对恶意 URL 进行检测。通过应用卷积神经网络来学习 URL 字符和单词的嵌入，该方法允许模型捕捉不同类型的语义信息，并通过高级词嵌入来解决任务中出现的罕见词问题。在大规模数据集上进行的实验表明，该方法优于现有方法，可显著提高模型性能。