Abstract:
Due to the continuous expansion of the scale of Web applications, the time cost of Web vulnerabilities scanning is increasing constantly. Therefore, this paper proposes a crawling method for efficient Web vulnerabilities scanning. Based on traditional crawler for Web vulnerabilities scanning, this method grouped the Web pages in phases by using the algorithm of incrementally frequent closed itemset mining, and built the page classification model based on page clusters and crawling record to filter the redundant pages created by the same service handler. The experiments show that the proposed method can reduce the time spent on website path traversal and page clustering, which could improve the efficiency of web vulnerabilities scanning.