According to Dey et al. [10], a good web crawler should contain the following features: a high performance system
architecture that can retrieve a large number of web pages at the same time, capability of dealing with memory stack
overflow resulted from large web page contents, decision on which page is next to be downloaded (ranking algorithms),
and strong system with existing resources and web servers against crashes.