With the rapid development of Internet technology, people have more and more
access to a variety of web page resources. At the same time, the current rapid
development of deep learning technology is often inseparable from the huge
amount of Web data resources. On the other hand, NLP is also an important part
of data processing technology, such as web page data extraction. At present,
the extraction technology of web page text mainly uses a single heuristic
function or strategy, and most of them need to determine the threshold
manually. With the rapid growth of the number and types of web resources, there
are still problems to be solved when using a single strategy to extract the
text information of different pages. This paper proposes a web page text
extraction algorithm based on multi-feature fusion. According to the text
information characteristics of web resources, DOM nodes are used as the
extraction unit to design multiple statistical features, and high-order
features are designed according to heuristic strategies. This method
establishes a small neural network, takes multiple features of DOM nodes as
input, predicts whether the nodes contain text information, makes full use of
different statistical information and extraction strategies, and adapts to more
types of pages. Experimental results show that this method has a good ability
of web page text extraction and avoids the problem of manually determining the
threshold.

本研究提出了一种基于多特征融合的网页文本提取算法，建立了小型神经网络，采用多种统计信息和提取策略，适应了更多页面类型，避免了手动确定阈值的问题。